aws

Tools to interact with elements of the Amazon Web Services (AWS).

Specifically, data scientists tend to interact heavily with S3 object storage.

class S3(location='eu-west-1', api_version=None, use_ssl=True, verify=True, endpoint_url=None, aws_account_id=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, **kwargs)[source]

Bases: ArgRepr

Wraps an S3 client for delayed instantiation and config encapsulation.

For more convenient function chaining in functional compositions, instances are also callable and simply return a new client when called.

Parameters:

location (str, optional) – The name of the region associated with the client. See the list of available ` locations <https://docs.aws.amazon.com/ global-infrastructure/latest/regions/aws-regions.html>`_ for choices. Defaults to eu-west-1 (Ireland).
api_version (str, optional) – The API version to use. By default, botocore will use the latest API version when creating a client. You only need to specify this parameter if you want to use a previous API version of the client. Defaults to None.
use_ssl (bool, optional) – Whether to use SSL. Defaults to True.
verify (bool or str, optional) – Whether to verify SSL certificates. Can be set to True, a path/to/cert/bundle.pem, or False. Defaults to True.
endpoint_url (str, optional) – The complete URL to use for the constructed client. Normally, botocore will automatically construct the appropriate URL to use when communicating with a service. You can specify a complete URL (including the “http/https” scheme) to override this behavior. If this value is provided, then use_ssl is ignored.
aws_account_id (str, optional) – The account id to use when creating the client. Defaults to None.
aws_access_key_id (str, optional) – The access key to use when creating the client. Defaults to None.
aws_secret_access_key (str, optional) – The secret key to use when creating the client. Defaults to None.
aws_session_token (str, optional) – The session token to use for the client. Defaults to None.
**kwargs – Additional parameters for the client. See the documentation of context and config for options. Defaults to None.

Raises:

AttributeError – If location is not a string.

__call__(*_, **__)[source]: New S3 client on every call, ignoring any (keyword) arguments.

property client: New S3 client on first request, cached for subsequent requests.

class S3Bucket(s3, bucket, location='eu-west-1', exists_ok=False, age=None)[source]

Bases: ArgRepr

Create/retrieve a bucket on/from S3-compatible object storage.

Parameters:

s3 (S3) – An instance of a wrapped S3 client.
bucket (str) – The (unique!) name of the bucket to create. May include any number of string placeholders (i.e., pairs of curly brackets) that will be interpolated when instances are called.
location (str, optional) – The physical datacenter location to create the bucket in. See the AWS documentation for options. Defaults to “eu-west-1”.
exists_ok (bool, optional) – Whether quietly return the requested bucket if it exists or raise an. exception. Defaults to False.
age (int, optional) – Defaults to None. If set, objects older than the specified number of days will be automatically deleted.

Raises:

AttributeError – If bucket or location are not strings.
TypeError – If age cannot be cast to an integer.
ValueError – If age is less than one.