Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Please generously STAR★ our project or donate to us!

## 🚀 Key Features

- **Configuration Management (`aloha.config`)**: Lazy-loaded settings (`SETTINGS`) using HOCON (Human-Optimized Config Object Notation), supporting environment profile overrides (`ENV_PROFILE` / `FILES_CONFIG`) and environment variable injection.
- **Configuration Management (`aloha.config`)**: Lazy-loaded settings (`SETTINGS`) using HOCON (Human-Optimized Config Object Notation), supporting environment profile overrides (`PROFILE_ENV` / `FILES_CONFIG`, with legacy `ENV_PROFILE` deprecated) and environment variable injection.
- **Concurrent-Safe Logging (`aloha.logger`)**: Multi-process safe daily rotating log file handler, console output, and automatic log paths configuration.
- **Database Operators (`aloha.db`)**: Pre-built SQLAlchemy-backed connections for PostgreSQL, MySQL, SQLite, DuckDB, MongoDB, Redis, Elasticsearch, and Kafka, with password resolution via a secure `PasswordVault` wrapper.
- **Encryption & Utilities (`aloha.encrypt`)**: Fast helpers for AES (ECB/CBC) encryption, RSA asymmetric key-pair generation/signatures, JWT encoding/decoding, and Base62 hashing.
Expand Down
10 changes: 7 additions & 3 deletions doc/en/README-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,18 @@

## OS environment variables

### `ENV_PROFILE`
### `PROFILE_ENV`

*Default value*: `None` (not defined).

Define the environment profile for the current process, such as `DEV | STG | PRD`.
This is usually used to decide which config file in `${DIR_CONFIG}` should be used as the entrypoint config.

If this environment variable is defined, `aloha` will first search for `main-${ENV_PROFILE}.conf`; otherwise it uses `main.conf`.
If this environment variable is defined, `aloha` will first search for `main-${PROFILE_ENV}.conf`; otherwise it uses `main.conf`.

> [!NOTE]
> **Backward Compatibility & Deprecation Notice**:
> If `PROFILE_ENV` is not defined or has no value, `aloha` falls back to reading the legacy `ENV_PROFILE` environment variable. If `ENV_PROFILE` is present, a `DeprecationWarning` will be emitted. `ENV_PROFILE` is deprecated and support will be removed in a future release. Please migrate to `PROFILE_ENV`.

### `ENTRYPOINT`

Expand Down Expand Up @@ -49,4 +53,4 @@ Define where to find configuration files.
*Default value*: `None` (not defined).

Optional. Define a comma-separated list of config files to load.
If this variable is set, `ENV_PROFILE` is ignored.
If this variable is set, `PROFILE_ENV` (and legacy `ENV_PROFILE`) is ignored.
6 changes: 3 additions & 3 deletions doc/en/api/util.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

## Time Utilities (`aloha.util.time`)

This module provides tools for wrapping function calls (such as HTTP requests via `requests` or `httpx`) with time constraints (timeouts), allowing execution of optional callbacks upon completion or failure.
This module provides tools for wrapping function calls (such as HTTP requests via `httpx2`) with time constraints (timeouts), allowing execution of optional callbacks upon completion or failure.

### Key Functions
- `run_with_timeout`: Wrap a synchronous function call with a timeout.
Expand All @@ -25,7 +25,7 @@ This module provides tools for wrapping function calls (such as HTTP requests vi
### Usage Example
```python
from aloha.util.time import run_with_timeout
import requests
import httpx2

def success_callback(response):
print("Request succeeded:", response.status_code)
Expand All @@ -36,7 +36,7 @@ def fail_callback(exception):
# Synchronous call with timeout
try:
run_with_timeout(
requests.get,
httpx2.get,
2.5, # 2.5 seconds timeout
"https://httpbin.org/delay/1",
fn_callback_success=success_callback,
Expand Down
2 changes: 1 addition & 1 deletion doc/skills/aloha_python/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ When developing Python code in this codebase, adhere to the following naming con

The `aloha` package is divided into several specialized sub-modules. See their respective reference documents for detailed APIs and usage examples:

- **[Configuration Management (`aloha.config`)](references/config.md)**: HOCON configuration loading, environment profile handling (`ENV_PROFILE`), and settings loading via `SETTINGS`.
- **[Configuration Management (`aloha.config`)](references/config.md)**: HOCON configuration loading, environment profile handling (`PROFILE_ENV`, legacy `ENV_PROFILE` is deprecated), and settings loading via `SETTINGS`.
- **[Logging Framework (`aloha.logger`)](references/logger.md)**: Safe concurrent multi-process logging, console handlers, and custom logger setups.
- **[Encryption & Hashing (`aloha.encrypt`)](references/encrypt.md)**: AES encryption, RSA keys generation & signing/verifying, JWT encoding/decoding, and dictionary/object hashing helpers.
- **[Testing Utilities (`aloha.testing`)](references/testing.md)**: Base `UnitTestCase` class with pre-configured settings/loggers, and `ServiceTestCase` for testing HTTP API endpoints.
Expand Down
10 changes: 6 additions & 4 deletions doc/skills/aloha_python/references/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ The configuration and startup behavior of an `aloha` application can be customiz

| Environment Variable | Default Value | Description |
| :------------------- | :----------------------- | :------------------------------------------------------------------------------------------------------------------------------- |
| `ENV_PROFILE` | `None` (undefined) | Specifies the running profile (e.g., `DEV`, `STG`, `PRD`). Determines the configuration entry file (`main-${ENV_PROFILE}.conf`). |
| `PROFILE_ENV` | `None` (undefined) | Specifies the running profile (e.g., `DEV`, `STG`, `PRD`). Determines the configuration entry file (`main-${PROFILE_ENV}.conf`). Falls back to legacy `ENV_PROFILE` with a deprecation warning if unset. `ENV_PROFILE` is deprecated and will be removed in a future release. |
| `ENTRYPOINT` | `None` (undefined) | Specifies the default Python module entry point when running `aloha start`. The module must contain a `main()` function. |
| `APP_MODULE` | `default` | Identifies the application/module name. Mapped to configuration key `APP_MODULE` and used as a prefix for the log file names. |
| `DIR_LOG` | `logs` | The directory where log files are stored. |
| `DIR_RESOURCE` | `resource` (under CWD) | Root directory containing non-code resources (e.g. assets, static data). |
| `DIR_CONFIG` | `${DIR_RESOURCE}/config` | Directory where configuration files are located. |
| `FILES_CONFIG` | `None` (undefined) | Comma-separated list of configuration filenames to load (e.g., `db.conf,server.conf`). If defined, it overrides `ENV_PROFILE`. |
| `FILES_CONFIG` | `None` (undefined) | Comma-separated list of configuration filenames to load (e.g., `db.conf,server.conf`). If defined, it overrides `PROFILE_ENV`. |

---

Expand All @@ -30,8 +30,10 @@ This module resolves paths for config directories, resource directories, and act
- `get_config_dir(*args) -> str`: Resolves the absolute path to the configuration directory. Relies on the `DIR_CONFIG` environment variable.
- `get_config_files() -> list`: Determines which HOCON configuration files should be loaded.
- If `FILES_CONFIG` environment variable is defined, it splits the list by comma and resolves their paths.
- If `FILES_CONFIG` is not defined but `ENV_PROFILE` is defined, it resolves `main-${ENV_PROFILE}.conf`.
- Otherwise, it defaults to `main.conf`.
- If `FILES_CONFIG` is not defined:
- Checks `PROFILE_ENV` first. If defined, it resolves `main-${PROFILE_ENV}.conf`.
- If `PROFILE_ENV` is not defined, falls back to legacy `ENV_PROFILE` with a `DeprecationWarning` (support for `ENV_PROFILE` will be removed in a future release).
- Otherwise (neither is defined), it defaults to `main.conf`.
- `get_project_base_dir(file_caller: str) -> str`: Traverses directories upwards from `file_caller` (typically passed as `__file__`) until it finds a directory containing no `__init__.py`, marking the project base root.

---
Expand Down
10 changes: 7 additions & 3 deletions doc/zh/README-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,18 @@

## 操作系统环境变量

### `ENV_PROFILE`
### `PROFILE_ENV`

*默认值*:`None`(未定义)。

用于指定当前进程运行环境,例如 `DEV | STG | PRD`。
通常用于决定 `${DIR_CONFIG}` 下哪个配置文件作为入口配置。

如果该变量已定义,`aloha` 会优先查找 `main-${ENV_PROFILE}.conf`;否则使用 `main.conf`。
如果该变量已定义,`aloha` 会优先查找 `main-${PROFILE_ENV}.conf`;否则使用 `main.conf`。

> [!NOTE]
> **向后兼容与弃用说明**:
> 如果 `PROFILE_ENV` 未定义或无值,`aloha` 会回退读取旧版环境变量 `ENV_PROFILE`。若检测到 `ENV_PROFILE` 有值,系统将输出 `DeprecationWarning` 弃用警告。`ENV_PROFILE` 已被弃用,并且在将来版本中会正式取消支持。请尽快迁移使用 `PROFILE_ENV`。

### `ENTRYPOINT`

Expand Down Expand Up @@ -49,4 +53,4 @@
*默认值*:`None`(未定义)。

可选项。用于定义以英文逗号分隔的配置文件列表。
如果该变量存在,则会忽略 `ENV_PROFILE`。
如果该变量存在,则会忽略 `PROFILE_ENV`(及旧版 `ENV_PROFILE`
6 changes: 3 additions & 3 deletions doc/zh/api/util.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

## 时间工具 (`aloha.util.time`)

该模块提供用于包装函数调用(如通过 `requests` 或 `httpx` 发起外部 HTTP 请求)的超时控制工具,并在操作成功或失败(超时/异常)时触发可选的回调函数。
该模块提供用于包装函数调用(如通过 `httpx2` 发起外部 HTTP 请求)的超时控制工具,并在操作成功或失败(超时/异常)时触发可选的回调函数。

### 核心函数
- `run_with_timeout`: 以同步方式运行函数,并应用超时限制。
Expand All @@ -25,7 +25,7 @@
### 使用示例
```python
from aloha.util.time import run_with_timeout
import requests
import httpx2

def success_callback(response):
print("请求成功:", response.status_code)
Expand All @@ -36,7 +36,7 @@ def fail_callback(exception):
# 同步超时包装调用
try:
run_with_timeout(
requests.get,
httpx2.get,
2.5, # 2.5 秒超时限制
"https://httpbin.org/delay/1",
fn_callback_success=success_callback,
Expand Down
6 changes: 2 additions & 4 deletions pkg/aloha/config/hocon.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,7 @@ def patched_fixup_self_references(cls, config, accept_unresolved=False):
if prop_path[0] == key:
if isinstance(previous_item, ConfigValues) and not accept_unresolved:
raise ConfigSubstitutionException(
"Property {variable} cannot be substituted. Check for cycles.".format(
variable=substitution.variable
)
f"Property {substitution.variable} cannot be substituted. Check for cycles."
)
else:
value = previous_item if len(prop_path) == 1 else previous_item.get(".".join(prop_path[1:]))
Expand Down Expand Up @@ -76,7 +74,7 @@ def load_config_from_hocon_files(config_files: list, base_dir: str):
"""
s = []
for config_file in config_files:
f = 'include required("%s")' % config_file
f = f'include required("{config_file}")'
s.append(f)
f = "\n".join(s)

Expand Down
34 changes: 26 additions & 8 deletions pkg/aloha/config/paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import sys
import warnings

__all__ = ("get_resource_dir", "get_config_dir", "get_current_module_dir", "get_project_base_dir", "path_join")
__all__ = ("get_config_dir", "get_current_module_dir", "get_project_base_dir", "get_resource_dir", "path_join")


def path_join(*args) -> str:
Expand Down Expand Up @@ -60,27 +60,45 @@ def get_config_files() -> list:
1. The function will look up the `FILES_CONFIG` environment variable to get a list of file names seperated by comma, if specified;
2. In case `FILES_CONFIG` is not specified, the function will use the default config file;
3. The default config file is determined by:
(a) If environment variable `ENV_PROFILE` is defined, the entry file will be "main-{ENV_PROFILE}.conf"
(b) If environment variable `ENV_PROFILE` is not defined, the entry config file will be "main.conf".
(a) If environment variable `PROFILE_ENV` is defined, the entry file will be "main-{PROFILE_ENV}.conf"
(b) If `PROFILE_ENV` is not defined, fallback to `ENV_PROFILE` (deprecated, support will be removed in a future release);
(c) If neither is defined, the entry config file will be "main.conf".
:return: list of string, which are file names of config files
"""
files_config = os.environ.get("FILES_CONFIG", None)
if files_config is None:
env_profile = os.environ.get("ENV_PROFILE", None)
if env_profile is None:
profile_env = os.environ.get("PROFILE_ENV")
if profile_env is not None and len(profile_env.strip()) == 0:
profile_env = None

if profile_env is None:
env_profile = os.environ.get("ENV_PROFILE")
if env_profile is not None and len(env_profile.strip()) == 0:
env_profile = None

if env_profile is not None:
warnings.warn(
"The 'ENV_PROFILE' environment variable is deprecated and will be removed in a future release. "
"Please use 'PROFILE_ENV' instead.",
DeprecationWarning,
stacklevel=2,
)
profile_env = env_profile

if profile_env is None:
files_config = "main.conf"
else:
files_config = "main-%s.conf" % env_profile
files_config = f"main-{profile_env}.conf"

files = files_config.split(",")
ret = []
msgs = []
for f in files:
file = get_config_dir(f)
if not os.path.exists(file):
msgs.append("Expecting config file [%s] but it does not exists!" % file)
msgs.append(f"Expecting config file [{file}] but it does not exists!")
else:
print(" ---> Loading config file [%s]" % file, file=sys.stderr)
print(f" ---> Loading config file [{file}]", file=sys.stderr)
ret.append(os.path.expandvars(f))
if len(ret) == 0:
msgs.append("No config files set properly, EMPTY config will be used!")
Expand Down
14 changes: 8 additions & 6 deletions pkg/aloha/db/base.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from typing import ClassVar

from ..encrypt import vault
from ..logger import LOG
from ..settings import SETTINGS
Expand All @@ -10,7 +12,7 @@
Caches vault instances for performance.
"""

_dict_cache_vault = {}
_dict_cache_vault: ClassVar[dict] = {}

@staticmethod
def get_vault(vault_type: str | None = None, vault_config: dict | None = None, **kwargs) -> vault.BaseVault:
Expand All @@ -32,7 +34,7 @@
encryption_method = vault_type or SETTINGS.config.get("PASSWORD_ENCRYPTION")
LOG.debug("Using password vault: %s", encryption_method) # nosemgrep

cache_key = "%s:%s" % (encryption_method, str(vault_config))
cache_key = f"{encryption_method}:{vault_config!s}"
if cache_key not in PasswordVault._dict_cache_vault:
if encryption_method in ("plain", "aes") or encryption_method is True:
v = vault.AesVault(**(vault_config or {}))
Expand All @@ -42,7 +44,7 @@
raise RuntimeError("Missing [CYBERARK_CONFIG] in config!")
v = vault.CyberArkVault(**config_cyberark)
else:
msg = "Using plain password vault as unknown value of PASSWORD_ENCRYPTION=%s in config." % encryption_method
msg = f"Using plain password vault as unknown value of PASSWORD_ENCRYPTION={encryption_method} in config."
LOG.info(msg) # nosemgrep
v = vault.DummyVault(**(vault_config or {}))
PasswordVault._dict_cache_vault[cache_key] = v
Expand All @@ -59,11 +61,11 @@
import sys

config_key = sys.argv[-1]
LOG.debug("Getting pwd for deploy key [deploy.%s]" % config_key)
LOG.debug(f"Getting pwd for deploy key [deploy.{config_key}]")
try:
db_config = SETTINGS.config["deploy"][config_key]
password_vault = PasswordVault.get_vault()
p = password_vault.get_password(db_config.get("password"))
LOG.debug("Decrypted PWD: %s" % p)
LOG.debug(f"Decrypted PWD: {p}")
except KeyError:
LOG.error("Please make sure config key [deploy.%s] exists!" % config_key)
LOG.error(f"Please make sure config key [deploy.{config_key}] exists!")
10 changes: 5 additions & 5 deletions pkg/aloha/db/duckdb.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

__all__ = ("DuckOperator",)

LOG.debug("duckdb version = %s, duckdb_engine = %s " % (duckdb.__version__, duckdb_engine.__version__))
LOG.debug(f"duckdb version = {duckdb.__version__}, duckdb_engine = {duckdb_engine.__version__} ")


class DuckOperator:
Expand Down Expand Up @@ -57,7 +57,7 @@ def __init__(self, db_config, **kwargs):
LOG.debug(msg)
except Exception as e:
LOG.exception(e)
raise RuntimeError("Failed to connect to DuckDB")
raise RuntimeError("Failed to connect to DuckDB") from e

def _prepare_database(self):
"""Prepare the database file and its parent directory."""
Expand All @@ -72,7 +72,7 @@ def _prepare_database(self):
parent_dir.mkdir(parents=True, exist_ok=True)
LOG.debug(f"Created directory: {parent_dir}")
except Exception as e:
raise RuntimeError(f"Failed to create directory '{parent_dir}': {e}")
raise RuntimeError(f"Failed to create directory '{parent_dir}': {e}") from e

if not path_obj.exists():
if self._config["read_only"]:
Expand All @@ -81,7 +81,7 @@ def _prepare_database(self):
LOG.debug(f"Database file not found, creating: {path}")
duckdb.connect(path).close()
except Exception as e:
raise RuntimeError(f"Failed to create database file '{path}': {e}")
raise RuntimeError(f"Failed to create database file '{path}': {e}") from e

def _initialize_schema(self):
"""Create or select the requested schema."""
Expand All @@ -101,7 +101,7 @@ def _initialize_schema(self):

self.engine.connect().execute(text(f"SET schema '{self._config['schema']}'"))
except Exception as e:
raise RuntimeError(f"Failed to initialize schema: {e}")
raise RuntimeError(f"Failed to initialize schema: {e}") from e

@property
def connection(self):
Expand Down
2 changes: 1 addition & 1 deletion pkg/aloha/db/elasticsearch.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ def build_index(self, index_name=None, index_config=None, raise_if_exist=False):
res = self.es.indices.create(index=index_name or self.index_name, body=index_config or self.index_config)
return res
else:
msg = "Index [%s] already exits" % self.index_name
msg = f"Index [{self.index_name}] already exits"
if raise_if_exist:
raise RuntimeError(msg)
else:
Expand Down
Loading