config_wrangler.config_templates.sqlalchemy_database module

pydantic model config_wrangler.config_templates.sqlalchemy_database.SQLAlchemyDatabase[source]

Bases: Credentials

Config:

  • validate_default: bool = True

  • validate_assignment: bool = True

  • validate_credentials: bool = True

Fields:
Validators:
field arraysize: int | None = 5000

Only used for Oracle. Passed to SQLAlchemy create_engine function.

Validated by:
field aws_access_key_id: str | None = None

For Amazon Redshift and with use_get_cluster_credentials = True only. If provided, use this access key to authenticate for the use_get_cluster_credentials call. If not provided, the user_id value will be used instead.

Validated by:
field aws_secret_access_key: str | None = None

For Amazon Redshift and with use_get_cluster_credentials = True only. If provided, use this access key to authenticate for the use_get_cluster_credentials call. If not provided, get_password() will be used to get the password from the configured password source (see PasswordSource for options.)

Validated by:
field create_engine_args: Dict[str, Any] = {}

A dictionary with extra arguments to pass to the SQLAlchemy create_engine function. See https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine

Validated by:
field database_name: str [Required]

https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.URL.database

Validated by:
field dialect: str [Required]

The SQLAlchemy dialect to use. See https://docs.sqlalchemy.org/en/20/dialects/

Validated by:
field driver: str | None = None

The python database driver to use. This is optional in the SQLAlchemy connection string it would be the optional +driver value that can come after the dialect name.

For example:

dialect=postgresql
driver=psycopg2

Would generate a connection string like:

postgresql+psycopg2://user:password@host:port/dbname

You could also provide both dialect and driver in the dialect field like this:

dialect=postgresql+psycopg2
Validated by:
field encoding: str | None = None

Passed to SQLAlchemy create_engine function.

Validated by:
field host: str | None = None

Hostname or IP number. May also be a data source name for some drivers. https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.URL.host

Validated by:
field keepass: KeepassConfig | None = None

If the password_source is KEEPASS, then load a sub-section with the config_wrangler.config_templates.keepass_config.KeepassConfig) settings

Validated by:
field keepass_config: str | None = 'keepass'

If the password_source is KEEPASS, then which root level config item contains the settings for Keepass (must be an instance of config_wrangler.config_templates.keepass_config.KeepassConfig)

Validated by:
field keepass_group: str | None = None

If the password_source is KEEPASS, which group in the Keepass database should be searched for an entry with a matching entry.

If is None, then the KeepassConfig.default_group value will be checked. If that is also None, then a ValueError will be raised.

Validated by:
field keepass_title: str | None = None

If the password_source is KEEPASS, this is an optional filter on the title of the keepass entries in the group.

Validated by:
field keyring_section: str | None = None

If the password_source is KEYRING, then which section (AKA system) should this module look for the password in.

See https://pypi.org/project/keyring/ or https://github.com/jaraco/keyring

Validated by:
field password_source: PasswordSourceValidated | None = None

The source to use when getting a password for the user. See PasswordSource for valid values.

Validated by:
field poolclass: str | None = None

Pool subclass that should be passed to SQLAlchemy create_engine function. Only supports None, QueuePool or NullPool. See https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.poolclass

Validated by:
field port: int | None = None

Integer port number. https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.URL.port

Validated by:
field raw_password: str | None = None

This is only used for the extremely non-secure CONFIG_FILE password source. The password is stored directly in the config file next to the user_id with the setting name raw_password

Validated by:
field rs_auto_create: bool | None = None

For Amazon Redshift and with use_get_cluster_credentials = True only. The value here is passed to the boto3 get_cluster_credentials call.

Validated by:
field rs_cluster_id: str | None = None

For Amazon Redshift only. The cluster name to use.

Validated by:
field rs_db_groups: List[str] | None = None

For Amazon Redshift and with use_get_cluster_credentials = True only. The value here is passed to the boto3 get_cluster_credentials call.

Validated by:
field rs_db_user_id: str | None = None

For Amazon Redshift and with use_get_cluster_credentials = True only. The value here is passed to the boto3 get_cluster_credentials call.

Validated by:
field rs_duration_seconds: int | None = 3600

For Amazon Redshift and with use_get_cluster_credentials = True only. The value here is passed to the boto3 get_cluster_credentials call.

Validated by:
field rs_new_credentials_seconds: int = 1800

For Amazon Redshift and with use_get_cluster_credentials = True only. How long should we wait before requesting new temporary credentials last (in seconds). This should be less than the value used in rs_duration_seconds. Note: The server might respond with an even shorter duration, if so that will be used.

Validated by:
field rs_region_name: str | None = None

For Amazon Redshift only. The region_name to use.

Validated by:
field use_get_cluster_credentials: bool = False

For Amazon Redshift only. Should the boto3 call be performed to get temporary database credentials? If so, then rs_db_user_id is required.

Validated by:
field user_id: str | None = None

The user ID to use

Validated by:
field validate_password_on_load: bool = True

Should config_wrangler query the password source for this password at load (startup) time? If so, it will raise an error if the password is None or an empty string. It does not actually connect or authenticate the user_id & password combination.

Validated by:
__init__(**data: Any) → None

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Uses something other than self the first arg to allow “self” as a settable attribute

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

validator check_model  »  all fields[source]
connect() → sqlalchemy.engine.base.Connection[source]

Connect to the configured database. Special handling for sqlite where subsequent calls will return the same connection.

classmethod construct(_fields_set: set[str] | None = None, **values: Any) → Model
copy(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, update: Dict[str, Any] | None = None, deep: bool = False) → Model

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`py data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include – Optional set or mapping specifying which fields to include in the copied model.

  • exclude – Optional set or mapping specifying which fields to exclude in the copied model.

  • update – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep – If True, the values of fields that are Pydantic models will be deep-copied.

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: IncEx = None, exclude: IncEx = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → Dict[str, Any]
classmethod from_orm(obj: Any) → Model
full_item_name(item_name: str = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get(section, item, fallback=Ellipsis)

Used as a drop in replacement for ConfigParser.get() with dynamic config field names (using a string variable for the section and item names instead of python code attribute access)

Warning

With this method Python code checkers (linters) will not warn about invalid config items. You can end up with runtime AttributeError errors.

get_cluster_credentials() → Tuple[str, str][source]
get_copy(copied_by: str = 'get_copy') → ConfigHierarchy

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_engine() → Engine[source]
get_list(section, item, fallback=Ellipsis) → list

Used as a drop in replacement for ConfigParser.get() + list parsing with dynamic config field names (using a string variable for the section and item names instead of python code attribute access) that is then parsed as a list.

Warning

With this method Python code checkers (linters) will not warn about invalid config items. You can end up with runtime AttributeError errors.

get_password() → str[source]

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_uri() → URL[source]
getboolean(section, item, fallback=Ellipsis) → bool

Used as a drop in replacement for ConfigParser.getboolean() with dynamic config field names (using a string variable for the section and item names instead of python code attribute access)

Warning

With this method Python code checkers (linters) will not warn about invalid config items. You can end up with runtime AttributeError errors.

json(*, include: IncEx = None, exclude: IncEx = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any) → str
classmethod model_construct(_fields_set: set[str] | None = None, **values: Any) → Model

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

Parameters:

  • _fields_set – The set of field names accepted for the Model instance.

  • values – Trusted or pre-validated data dictionary.

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: dict[str, Any] | None = None, deep: bool = False) → Model

Usage docs: https://docs.pydantic.dev/2.6/concepts/serialization/#model_copy

Returns a copy of the model.

Parameters:

  • update – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep – Set to True to make a deep copy of the model.

Returns:

New model instance.

model_dump(*, mode: Literal['json', 'python'] | str = 'python', include: IncEx = None, exclude: IncEx = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, round_trip: bool = False, warnings: bool = True) → dict[str, Any]

Usage docs: https://docs.pydantic.dev/2.6/concepts/serialization/#modelmodel_dump

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Parameters:

  • mode – The mode in which to_python should run. If mode is ‘json’, the output will only contain JSON serializable types. If mode is ‘python’, the output may contain non-JSON-serializable Python objects.

  • include – A list of fields to include in the output.

  • exclude – A list of fields to exclude from the output.

  • by_alias – Whether to use the field’s alias in the dictionary key if defined.

  • exclude_unset – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults – Whether to exclude fields that are set to their default value.

  • exclude_none – Whether to exclude fields that have a value of None.

  • round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings – Whether to log warnings when invalid fields are encountered.

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, include: IncEx = None, exclude: IncEx = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, round_trip: bool = False, warnings: bool = True) → str

Usage docs: https://docs.pydantic.dev/2.6/concepts/serialization/#modelmodel_dump_json

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • include – Field(s) to include in the JSON output.

  • exclude – Field(s) to exclude from the JSON output.

  • by_alias – Whether to serialize using field aliases.

  • exclude_unset – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults – Whether to exclude fields that are set to their default value.

  • exclude_none – Whether to exclude fields that have a value of None.

  • round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings – Whether to log warnings when invalid fields are encountered.

Returns:

A JSON string representation of the model.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] = None) → dict[str, Any]
classmethod model_json_schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}', schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'pydantic.json_schema.GenerateJsonSchema'>, mode: ~typing.Literal['validation', 'serialization'] = 'validation') → dict[str, Any]

Generates a JSON schema for a model class.

Parameters:

  • by_alias – Whether to use attribute aliases or not.

  • ref_template – The reference template.

  • schema_generator – To override the logic used to generate the JSON schema, as a subclass of GenerateJsonSchema with your desired modifications

  • mode – The mode in which to generate the schema.

Returns:

The JSON schema for the given model class.

classmethod model_parametrized_name(params: tuple[type[Any], ...]) → str

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_ModelMetaclass__context: Any) → None

We need to both initialize private attributes and call the user-defined model_post_init method.

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: dict[str, Any] | None = None) → bool | None

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors – Whether to raise errors, defaults to True.

  • _parent_namespace_depth – The depth level of the parent namespace, defaults to 2.

  • _types_namespace – The types namespace, defaults to None.

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, from_attributes: bool | None = None, context: dict[str, Any] | None = None) → Model

Validate a pydantic model instance.

Parameters:

  • obj – The object to validate.

  • strict – Whether to enforce types strictly.

  • from_attributes – Whether to extract data from object attributes.

  • context – Additional context to pass to the validator.

Raises:

ValidationError – If the object could not be validated.

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, context: dict[str, Any] | None = None) → Model

Usage docs: https://docs.pydantic.dev/2.6/concepts/json/#json-parsing

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data – The JSON data to validate.

  • strict – Whether to enforce types strictly.

  • context – Extra variables to pass to the validator.

Returns:

The validated Pydantic model.

Raises:

ValueError – If json_data is not a JSON string.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, context: dict[str, Any] | None = None) → Model

Validate the given object contains string data against the Pydantic model.

Parameters:

  • obj – The object contains string data to validate.

  • strict – Whether to enforce types strictly.

  • context – Extra variables to pass to the validator.

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Model
classmethod parse_obj(obj: Any) → Model
classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Model
raw_connection()[source]

Return a “raw” DBAPI connection from the connection pool.

See https://docs.sqlalchemy.org/en/20/core/connections.html#sqlalchemy.engine.Engine.raw_connection

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}') → Dict[str, Any]
classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any) → str
session() → Session[source]

Build a SQLAlchemy session from the configured database.

set_as_child(name: str, other_config_item: ConfigHierarchy)
validator translate  »  all fields[source]
static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

classmethod update_forward_refs(**localns: Any) → None
classmethod validate(value: Any) → Model
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

property model_extra: dict[str, Any] | None

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.