mirror of
https://github.com/dtomlinson91/panaetius.git
synced 2025-12-22 04:55:44 +00:00
135 lines
3.3 KiB
Markdown
135 lines
3.3 KiB
Markdown
# Panaetius
|
||
|
||
|
||
|
||
|
||
This package provides:
|
||
|
||
- Functionality to read user variables from a `config.yml` or environment variables.
|
||
- A convenient default logging formatter printing `json` that can save to disk and rotate.
|
||
- Utility functions.
|
||
|
||
## Config
|
||
|
||
### options
|
||
|
||
#### skip_header_init
|
||
|
||
If `skip_header_init=True` then the `config_path` will not use the `header_variable` as the
|
||
sub-directory in the `config_path`.
|
||
|
||
E.g
|
||
|
||
`CONFIG = panaetius.Config("tembo", "~/tembo/.config", skip_header_init=True)`
|
||
|
||
Will look in `~/tembo/config/config.yml`.
|
||
|
||
If `skip_header_init=False` then would look in `~/tembo/config/tembo/config.yml`.
|
||
|
||
### Module
|
||
|
||
Convenient to place in a package/sub-package `__init__.py`.
|
||
|
||
See Tembo for an example: <https://github.com/tembo-pages/tembo-core/blob/main/tembo/cli/__init__.py>
|
||
|
||
Example snippet to use in a module:
|
||
|
||
```python
|
||
"""Subpackage that contains the CLI application."""
|
||
|
||
import os
|
||
from typing import Any
|
||
|
||
import panaetius
|
||
from panaetius.exceptions import LoggingDirectoryDoesNotExistException
|
||
|
||
|
||
if (config_path := os.environ.get("TEMBO_CONFIG")) is not None:
|
||
CONFIG: Any = panaetius.Config("tembo", config_path, skip_header_init=True)
|
||
else:
|
||
CONFIG = panaetius.Config(
|
||
"tembo", "~/tembo/.config", skip_header_init=True
|
||
)
|
||
|
||
|
||
panaetius.set_config(CONFIG, "base_path", "~/tembo")
|
||
panaetius.set_config(CONFIG, "template_path", "~/tembo/.templates")
|
||
panaetius.set_config(CONFIG, "scopes", {})
|
||
panaetius.set_config(CONFIG, "logging.level", "DEBUG")
|
||
panaetius.set_config(CONFIG, "logging.path")
|
||
|
||
try:
|
||
logger = panaetius.set_logger(
|
||
CONFIG, panaetius.SimpleLogger(logging_level=CONFIG.logging_level)
|
||
)
|
||
except LoggingDirectoryDoesNotExistException:
|
||
_LOGGING_PATH = CONFIG.logging_path
|
||
CONFIG.logging_path = ""
|
||
logger = panaetius.set_logger(
|
||
CONFIG, panaetius.SimpleLogger(logging_level=CONFIG.logging_level)
|
||
)
|
||
logger.warning("Logging directory %s does not exist", _LOGGING_PATH)
|
||
|
||
```
|
||
|
||
This means in `./tembo/cli/cli.py` you can
|
||
|
||
```python
|
||
import tembo.cli
|
||
|
||
# access the CONFIG instance + variables from the config.yml
|
||
tembo.cli.CONFIG
|
||
```
|
||
|
||
### Script
|
||
|
||
#### with config file
|
||
|
||
Create `./config/config.yml` in the same directory as the script.
|
||
|
||
In the script initialise a `CONFIG` object:
|
||
|
||
```python
|
||
import pathlib
|
||
|
||
import panaetius
|
||
|
||
CONFIG = panaetius.Config(
|
||
"teenagers_scraper", str(pathlib.Path(__file__).parents[0] / ".config"), skip_header_init=True
|
||
)
|
||
```
|
||
|
||
Set variables in the same way as the module above.
|
||
|
||
#### with env vars only
|
||
|
||
```python
|
||
import panaetius
|
||
|
||
CONFIG = panaetius.Config("mart")
|
||
panaetius.set_config(CONFIG, "username")
|
||
panaetius.set_config(CONFIG, "password")
|
||
```
|
||
|
||
#### quickstart logging
|
||
|
||
```python
|
||
import panaetius
|
||
|
||
|
||
def get_logger():
|
||
logging_dir = pathlib.Path(__file__).parents[0] / "logs"
|
||
logging_dir.mkdir(parents=True, exist_ok=True)
|
||
|
||
CONFIG = panaetius.Config("training_data_into_gcp", skip_header_init=True)
|
||
panaetius.set_config(CONFIG, "logging.level", "DEBUG")
|
||
panaetius.set_config(CONFIG, "logging.path", logging_dir)
|
||
return panaetius.set_logger(CONFIG, panaetius.SimpleLogger(logging_level=CONFIG.logging_level))
|
||
```
|
||
|
||
## Utility Functions
|
||
|
||
### Squasher
|
||
|
||
Squashes a json object or Python dictionary into a single level dictionary.
|