brokenspoke_analyzer.core.exporter module¶

Define functions to export the data to various destinations.

class brokenspoke_analyzer.core.exporter.Exporter(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)¶

Bases: str, Enum

Define the available exporters.

local = 'local'¶

none = 'none'¶

s3 = 's3'¶

s3_custom = 's3_custom'¶

brokenspoke_analyzer.core.exporter.auto_export(export_dir, tables, database_url)¶

Export PostgreSQL/PostGIS tables to their repective files.

Regular tables are exported into CSV files. GIS tables are exported either to geojson or sometimes shapefiles (or both).

Return type:: None

brokenspoke_analyzer.core.exporter.bundle(src_dir)¶

Bundle the content of src_dir into a zip file and save it into src_dir.

Return type:: Path

brokenspoke_analyzer.core.exporter.calver_base(country, city, region=None, date_override=None, base_dir=PosixPath('.'))¶

Build the base part of the calver path.

Return type:: Path

brokenspoke_analyzer.core.exporter.calver_revision(dirs)¶

Build the revision part of the calver path.

Return type:: int

Examples:

>>> dirs=[pathlib.Path('usa/new mexico/santa rosa/23.08')]
>>> assert calver_revision(dirs) == 1
>>> dirs.append(pathlib.Path('usa/new mexico/santa rosa/23.08.1'))
>>> assert calver_revision(dirs) == 2

brokenspoke_analyzer.core.exporter.create_calver_directories(country, city, region, date_override=None, base_dir=PosixPath('.'))¶

Create a directory structure following calver to export the tables.

The calver scheme is based and inspired by the BNA mechanics standards: <country>/<egion>/<city>/YY.MM[.Micro] See https://calver.org/#scheme for more details.

Examples: * usa/tx/austin/23.08 * usa/tx/austin/23.12.2 * spain/valencia/valencia/23.08 :rtype: Path

>>> today = date.today()
>>> calver = f"{today.strftime('%y.%m')}"
>>> directory = create_calver_directories("usa", "austin", "tx")
>>> assert directory == pathlib.Path(f"usa/tx/austin/{calver}")

brokenspoke_analyzer.core.exporter.create_calver_s3_directories(bucket_name, country, city, region=None)¶

Create the calver directory in the S3 bucket.

Return type:: Path

brokenspoke_analyzer.core.exporter.export_to_csv(export_dir, tables, engine)¶

Export a list of PostgreSQL tables to CSV files.

Return type:: None

brokenspoke_analyzer.core.exporter.export_to_geojson(export_dir, tables, database_url)¶

Export a list of PostGIS tables to GeoJSON files.

Return type:: None

brokenspoke_analyzer.core.exporter.export_to_shp(export_dir, tables, database_url)¶

Export a list of PostGIS tables to Shapefiles.

Return type:: None

brokenspoke_analyzer.core.exporter.local_files(database_url, export_dir, with_bundle=False)¶

Export result files into a local directory.

Return type:: None

brokenspoke_analyzer.core.exporter.s3(database_url, bucket_name, folder=PosixPath('.'), with_bundle=False)¶

Export PostgreSQL/PostGIS tables to an S3 Bucket.

Return type:: None

brokenspoke_analyzer.core.exporter.s3_directories(bucket_name, s3_dir=PosixPath('.'))¶

Create a custom directory in the S3 bucket.

Return type:: Path