brokenspoke_analyzer.core.exporter module

Define functions to export the data to various destinations.

class brokenspoke_analyzer.core.exporter.Exporter(value, names=None, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Define the available exporters.

local = 'local'

none = 'none'

s3 = 's3'

brokenspoke_analyzer.core.exporter.auto_export(export_dir, tables, database_url)

Export PostgreSQL/PostGIS tables to their repective files.

Regular tables are exported into CSV files. GIS tables are exported either to geojson or sometimes shapefiles (or both).

Return type:

None

brokenspoke_analyzer.core.exporter.calver_base(country, city, region=None, date_override=None, base_dir=PosixPath('.'))

Build the base part of the calver path.

Return type:

Path

brokenspoke_analyzer.core.exporter.calver_revision(dirs)

Build the revision part of the calver path.

Return type:

int

Examples:

>>> dirs=[pathlib.Path('usa/new mexico/santa rosa/23.10')]
>>> assert calver_revision(dirs) == 1
>>> dirs.append(pathlib.Path('usa/new mexico/santa rosa/23.10.1'))
>>> assert calver_revision(dirs) == 2

brokenspoke_analyzer.core.exporter.create_calver_directories(country, city, region, date_override=None, base_dir=PosixPath('.'))

Create a directory structure following calver to export the tables.

The calver scheme is based and inspired by the BNA mechanics standards: <country>/<egion>/<city>/YY.MM[.Micro] See https://calver.org/#scheme for more details.

Examples: * usa/tx/austin/23.8 * usa/tx/austin/23.12.2 * spain/valencia/valencia/23.8 :rtype: Path

>>> today = date.today()
>>> calver = f"{today.strftime('%y')}.{today.month}"
>>> directory = create_calver_directories("usa", "austin", "tx")
>>> assert directory == pathlib.Path(f"usa/tx/austin/{calver}")

brokenspoke_analyzer.core.exporter.create_calver_s3_directories(bucket_name, country, city, region=None)

Create the calver directory in the S3 bucket.

Return type:

Path

brokenspoke_analyzer.core.exporter.export_to_csv(export_dir, tables, engine)

Export a list of PostgreSQL tables to CSV files.

Return type:

None

brokenspoke_analyzer.core.exporter.export_to_geojson(export_dir, tables, database_url)

Export a list of PostGIS tables to GeoJSON files.

Return type:

None

brokenspoke_analyzer.core.exporter.export_to_shp(export_dir, tables, database_url)

Export a list of PostGIS tables to Shapefiles.

Return type:

None

brokenspoke_analyzer.core.exporter.s3(database_url, bucket_name, folder=PosixPath('.'))

Export PostgreSQL/PostGIS tables to an S3 Bucket.

Return type:

None

brokenspoke_analyzer.core.exporter.s3_directories(bucket_name, s3_dir)

Create a custom directory in the S3 bucket.

Return type:

Path