brokenspoke_analyzer.core.ingestor module

Define functions that will be use to ingest the data.

class brokenspoke_analyzer.core.ingestor.LODESPart(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Represent the part of the state file.

AUX = 'aux'

MAIN = 'main'

brokenspoke_analyzer.core.ingestor.all_wrapper(database_url, input_dir, country, city, region, fips_code='0', lodes_year=2019, buffer=2680)

Wrap the all the import_* functions.

Wrap the all the import_* functions to allow calling them with only parameters that cannot be computed.

Return type:

None

brokenspoke_analyzer.core.ingestor.delete_block_outside_buffer(engine, buffer)

Delete the blocks which are outside the boundaries+buffer.

Return type:

None

brokenspoke_analyzer.core.ingestor.delete_water_blocks(engine)

Delete the water blocks located within the city boundaries.

Return type:

None

brokenspoke_analyzer.core.ingestor.import_all(engine, country, output_srid, buffer, boundary_file, population_file, water_blocks_file, input_dir, osm_file, state_speed_limits_csv, city_speed_limits_csv, city_fips, state=None, lodes_year=None, city_speed_limit_override=None)

Import all the data.

Return type:

None

brokenspoke_analyzer.core.ingestor.import_and_transform_shapefile(engine, shapefile, table, output_srid, input_srid=4326)

Import a shapefile into PostGIS with shp2pgsql.

Return type:

None

brokenspoke_analyzer.core.ingestor.import_jobs(engine, state, lodes_year, input_dir)

Import all jobs from US census data.

This function is idempotent. The data will be recreated every time.

Return type:

None

brokenspoke_analyzer.core.ingestor.import_neighborhood(engine, country, boundary_file, population_file, water_blocks_file, output_srid, buffer)

Import neighborhood data.

This function is idempotent. The data will be recreated every time.

Return type:

None

brokenspoke_analyzer.core.ingestor.import_osm_data(engine, osm_file, output_srid, state_fips, city_fips, state_speed_limits_csv, city_speed_limits_csv, city_speed_limit_override=None)

Import data related to OSM.

Remark: can only be run afer import_neighborhood(). It requires some table to exist to compute the boundary box.

Return type:

None

brokenspoke_analyzer.core.ingestor.jobs_wrapper(database_url, input_dir, state_abbreviation, lodes_year)

Wrap the import_jobs function.

Wrap the import_jobs function to allow calling it with only parameters that cannot be computed.

Return type:

None

brokenspoke_analyzer.core.ingestor.load_jobs(engine, state, lodes_part, csvfile)

Load employment data from the US census website.

Return type:

None

brokenspoke_analyzer.core.ingestor.load_water_blocks(engine, csvfile)

Create the water blocks table and load the data into it from a CSV file.

Return type:

None

brokenspoke_analyzer.core.ingestor.manage_speed_limits(engine, state_fips, city_fips, state_speed_limits_csv, city_speed_limits_csv, city_speed_limit_override=None)

Manage the state and city speed limits..

Return type:

None

brokenspoke_analyzer.core.ingestor.move_tables(engine)

Move some tables to the “received” schema.

Return type:

None

brokenspoke_analyzer.core.ingestor.neighborhood_wrapper(database_url, input_dir, country, city, region, buffer)

Wrap the import_neighborhood .

Wrap the import_neighborhood function to allow calling it with only parameters that cannot be computed.

Return type:

None

brokenspoke_analyzer.core.ingestor.osm_wrapper(database_url, input_dir, country, city, region, fips_code)

Wrap the import_osm_data function.

Wrap the import_osm_data function to allow calling it with only parameters that cannot be computed.

Return type:

None

brokenspoke_analyzer.core.ingestor.rename_neighborhood_tables(engine)

Rename neighborhood tables.

Return type:

None

brokenspoke_analyzer.core.ingestor.retrieve_boundary_box(engine)

Retrieve the city boundary box.

Return type:

tuple[float, float, float, float]

brokenspoke_analyzer.core.ingestor.retrieve_city_speed_limit(engine, city_fips)

Retrieve the city speed limit from the imported speed limit data.

Return type:

str | None

brokenspoke_analyzer.core.ingestor.retrieve_default_speed_limits(engine)

Retrieve the state and city default speed limits.

Return type:

tuple[int | None, int | None]

brokenspoke_analyzer.core.ingestor.retrieve_population(engine)

Retrieve the population from the imported census data.

Return type:

int

brokenspoke_analyzer.core.ingestor.retrieve_speed_limit(engine, query)

Retrieve the speed limit from the imported speed limit data.

Return type:

str | None

brokenspoke_analyzer.core.ingestor.retrieve_state_speed_limit(engine, state_fips)

Retrieve the state speed limit from the imported speed limit data.

Return type:

str | None