network_wrangler.RoadwayNetwork

class network_wrangler.RoadwayNetwork(nodes, links, shapes=None, node_foreign_key=None, link_foreign_key=None, shape_foreign_key=None, unique_link_key=None, unique_node_key=None, unique_link_ids=None, unique_node_ids=None, crs=None, **kwargs)[source]

Bases: object

Representation of a Roadway Network.

Typical usage example:

net = RoadwayNetwork.read(
    link_filename=MY_LINK_FILE,
    node_filename=MY_NODE_FILE,
    shape_filename=MY_SHAPE_FILE,
    shape_foreign_key ='shape_id',
)
my_selection = {
    "link": [{"name": ["I 35E"]}],
    "A": {"osm_node_id": "961117623"},  # start searching for segments at A
    "B": {"osm_node_id": "2564047368"},
}
net.select_roadway_features(my_selection)

my_change = [
    {
        'property': 'lanes',
        'existing': 1,
        'set': 2,
     },
     {
        'property': 'drive_access',
        'set': 0,
      },
]

my_net.apply_roadway_feature_change(
    my_net.select_roadway_features(my_selection),
    my_change
)

ml_net = net.create_managed_lane_network(in_place=False)
ml_net.is_network_connected(mode="drive"))
_, disconnected_nodes = ml_net.assess_connectivity(mode="walk", ignore_end_nodes=True)
ml_net.write(filename=my_out_prefix, path=my_dir)
nodes_df

node data

Type:

GeoDataFrame

link data, including start and end nodes and associated shape

Type:

GeoDataFrame

shapes_df

detailed shape data

Type:

GeoDataFrame

crs

coordinate reference system, ESPG number

Type:

int

node_foreign_key

variable linking the node table to the link table

Type:

str

list of variable linking the link table to the node foreign key

Type:

list

shape_foreign_key

variable linking the links table and shape table

Type:

str

list of variables unique to each link

Type:

list

unique_node_ids

list of variables unique to each node

Type:

list

Mapping of modes to link variables in the network

Type:

dict

modes_to_network_nodes_variables

Mapping of modes to node variables in the network

Type:

dict

managed_lanes_node_id_scalar

Scalar values added to primary keys for nodes for corresponding managed lanes.

Type:

int

Scalar values added to primary keys for links for corresponding managed lanes.

Type:

int

managed_lanes_required_attributes

attributes that must be specified in managed lane projects.

Type:

list

keep_same_attributes_ml_and_gp

attributes to copy to managed lanes from parallel general purpose lanes.

Type:

list

selections

dictionary storing selections in case they are made repeatedly

Type:

dict

__init__(nodes, links, shapes=None, node_foreign_key=None, link_foreign_key=None, shape_foreign_key=None, unique_link_key=None, unique_node_key=None, unique_link_ids=None, unique_node_ids=None, crs=None, **kwargs)[source]

Constructor

Methods

__init__(nodes, links[, shapes, ...])

Constructor

add_incident_link_data_to_nodes([links_df, ...])

Add data from links going to/from nodes to node.

add_new_roadway_feature_change(links, nodes)

add the new roadway features defined in the project card.

addition_map(links, nodes)

Shows which links and nodes are added to the roadway network

apply(project_card_dictionary)

Wrapper method to apply a project to a roadway network.

apply_managed_lane_feature_change(link_idx, ...)

Apply the managed lane feature changes to the roadway network

apply_python_calculation(pycode[, in_place])

Changes roadway network object by executing pycode.

apply_roadway_feature_change(link_idx, ...)

Changes the roadway attributes for the selected features based on the project card information passed

assess_connectivity([mode, ...])

Returns a network graph and list of disconnected subgraphs as described by a list of their member nodes.

build_selection_key(selection_dict)

Selections are stored by a key combining the query and the A and B ids.

create_dummy_connector_links(ml_df[, ...])

create dummy connector links between the general purpose and managed lanes

create_managed_lane_network([...])

Create a roadway network with managed lanes links separated out.

delete_roadway_feature_change(links, nodes)

delete the roadway features defined in the project card.

deletion_map(links, nodes)

Shows which links and nodes are deleted from the roadway network

get_managed_lane_node_ids(nodes_list[, scalar])

Transform a list of node IDS by a scalar.

get_modal_graph(links_df, nodes_df[, mode, ...])

Determines if the network graph is "strongly" connected A graph is strongly connected if each vertex is reachable from every other vertex.

get_modal_links_nodes(links_df, nodes_df[, ...])

Returns nodes and link dataframes for specific mode.

get_property_by_time_period_and_group(prop)

Return a series for the properties with a specific group or time period.

identify_segment(O_id, D_id[, ...])

param endpoints:

list of length of two unique keys of nodes making up endpoints of segment

identify_segment_endpoints([mode, links_df, ...])

param mode:

list of modes of the network, one of drive,`transit`,

is_network_connected([mode, links_df, nodes_df])

Determines if the network graph is "strongly" connected A graph is strongly connected if each vertex is reachable from every other vertex.

load_transform_network(node_filename, ...[, ...])

Reads roadway network files from disk and transforms them into GeoDataFrames.

network_connection_plot(G, ...)

Plot a graph to check for network connection.

orig_dest_nodes_foreign_key(selection[, ...])

Returns the foreign key id (whatever is used in the u and v variables in the links file) for the AB nodes as a tuple.

ox_graph(nodes_df, links_df[, ...])

create an osmnx-flavored network graph

path_search(candidate_links_df, O_id, D_id)

param candidate_links:

selection of links geodataframe with links likely to be part of path

read(link_filename, node_filename, ...[, ...])

Reads a network from the roadway network standard Validates that it conforms to the schema

roadway_net_to_gdf(roadway_net)

rtype:

GeoDataFrame

select_roadway_features(selection[, ...])

Selects roadway features that satisfy selection criteria

selection_has_unique_link_id(selection_dict)

rtype:

bool

selection_map(selected_link_idx[, A, B, ...])

Shows which links are selected for roadway property change or parallel managed lanes category of roadway projects.

shortest_path(graph_links_df, O_id, D_id[, ...])

rtype:

tuple

update_distance([links_df, use_shapes, ...])

Calculate link distance in specified units to network variable using either straight line distance or (if specified) shape distance if available.

validate_link_schema(link_filename[, ...])

Validate roadway network data link schema and output a boolean

validate_node_schema(node_file[, ...])

Validate roadway network data node schema and output a boolean

validate_properties(properties[, ...])

If there are change or existing commands, make sure that that property exists in the network.

validate_selection(selection[, ...])

Evaluate whetther the selection dictionary contains the minimum required values.

validate_shape_schema(shape_file[, ...])

Validate roadway network data shape schema and output a boolean

validate_uniqueness()

Confirms that the unique identifiers are met.

write([path, filename])

Writes a network in the roadway network standard

Add data from links going to/from nodes to node.

Return type:

DataFrame

Parameters:

  • links_df – if specified, will assess connectivity of this links list rather than self.links_df

  • nodes_df – if specified, will assess connectivity of this nodes list rather than self.nodes_df

  • link_variables – list of columns in links dataframe to add to incident nodes

Returns:

nodes DataFrame with link data where length is N*number of links going in/out

add_new_roadway_feature_change(links, nodes)[source]

add the new roadway features defined in the project card. new shapes are also added for the new roadway links.

Return type:

None

Parameters:

  • links – list of dictionaries

  • nodes – list of dictionaries

returns: None

Todo

validate links and nodes dictionary

addition_map(links, nodes)[source]

Shows which links and nodes are added to the roadway network

apply(project_card_dictionary)[source]

Wrapper method to apply a project to a roadway network.

Parameters:

project_card_dictionary – dict a dictionary of the project card object

apply_managed_lane_feature_change(link_idx, properties, in_place=True)[source]

Apply the managed lane feature changes to the roadway network

Parameters:

  • link_idx – list of lndices of all links to apply change to

  • properties – list of dictionarys roadway properties to change

  • in_place – boolean to indicate whether to update self or return a new roadway network object

Todo

decide on connectors info when they are more specific in project card

apply_python_calculation(pycode, in_place=True)[source]

Changes roadway network object by executing pycode.

Parameters:

  • pycode – python code which changes values in the roadway network object

  • in_place – update self or return a new roadway network object

apply_roadway_feature_change(link_idx, properties, in_place=True)[source]

Changes the roadway attributes for the selected features based on the project card information passed

Parameters:

  • link_idx – list lndices of all links to apply change to

  • properties – list of dictionarys roadway properties to change

  • in_place – boolean update self or return a new roadway network object

assess_connectivity(mode='', ignore_end_nodes=True, links_df=None, nodes_df=None)[source]

Returns a network graph and list of disconnected subgraphs as described by a list of their member nodes.

Parameters:

  • mode – list of modes of the network, one of drive,`transit`, walk, bike

  • ignore_end_nodes – if True, ignores stray singleton nodes

  • links_df – if specified, will assess connectivity of this links list rather than self.links_df

  • nodes_df – if specified, will assess connectivity of this nodes list rather than self.nodes_df

Returns: Tuple of

Network Graph (osmnx flavored networkX DiGraph) List of disconnected subgraphs described by the list of their

member nodes (as described by their model_node_id)

build_selection_key(selection_dict)[source]

Selections are stored by a key combining the query and the A and B ids. This method combines the two for you based on the selection dictionary.

Return type:

tuple

Parameters:

selection_dictonary – Selection Dictionary

Returns: Tuple serving as the selection key.

create dummy connector links between the general purpose and managed lanes

Parameters:

  • gp_df – GeoDataFrame dataframe of general purpose links (where managed lane also exists)

  • ml_df – GeoDataFrame dataframe of corresponding managed lane links,

  • access_lanes – int number of lanes in access dummy link

  • egress_lanes – int number of lanes in egress dummy link

  • access_roadway – str roaday type for access dummy link

  • egress_roadway – str roadway type for egress dummy link

  • access_name_prefix – str prefix for access dummy link name

  • egress_name_prefix – str prefix for egress dummy link name

create_managed_lane_network(keep_same_attributes_ml_and_gp=None, keep_additional_attributes_ml_and_gp=[], managed_lanes_required_attributes=[], managed_lanes_node_id_scalar=None, managed_lanes_link_id_scalar=None, in_place=False)[source]

Create a roadway network with managed lanes links separated out. Add new parallel managed lane links, access/egress links, and add shapes corresponding to the new links

Return type:

RoadwayNetwork

Parameters:

  • keep_same_attributes_ml_and_gp – list of attributes to copy from general purpose lane to managed lane. If not specified, will look for value in the RoadwayNetwork instance. If not found there, will default to KEEP_SAME_ATTRIBUTES_ML_AND_GP.

  • keep_additional_attributes_ml_and_gp – list of additional attributes to add. This is useful if you want to leave the default attributes and then ALSO some others.

  • managed_lanes_required_attributes – list of attributes that are required to be specified in new managed lanes. If not specified, will look for value in the RoadwayNetwork instance. If not found there, will default to MANAGED_LANES_REQUIRED_ATTRIBUTES.

  • managed_lanes_node_id_scalar – integer value added to original node IDs to create managed lane unique ids. If not specified, will look for value in the RoadwayNetwork instance. If not found there, will default to MANAGED_LANES_NODE_ID_SCALAR.

  • managed_lanes_link_id_scalar – integer value added to original link IDs to create managed lane unique ids. If not specified, will look for value in the RoadwayNetwork instance. If not found there, will default to MANAGED_LANES_LINK_ID_SCALAR.

  • in_place – update self or return a new roadway network object

returns: A RoadwayNetwork instance

Todo

make this a more rigorous test

delete_roadway_feature_change(links, nodes, ignore_missing=True)[source]

delete the roadway features defined in the project card. valid links and nodes defined in the project gets deleted and shapes corresponding to the deleted links are also deleted.

Return type:

None

Parameters:

  • links – dict list of dictionaries

  • nodes – dict list of dictionaries

  • ignore_missing – bool If True, will only warn about links/nodes that are missing from network but specified to “delete” in project card If False, will fail.

deletion_map(links, nodes)[source]

Shows which links and nodes are deleted from the roadway network

static get_managed_lane_node_ids(nodes_list, scalar=4500000)[source]

Transform a list of node IDS by a scalar. ..todo #237 what if node ids are not a number?

Parameters:

  • nodes_list – list of integers

  • scalar – value to add to node IDs

Returns: list of integers

static get_modal_graph(links_df, nodes_df, mode=None, modes_to_network_link_variables={'bike': ['bike_access'], 'bus': ['bus_only', 'drive_access'], 'drive': ['drive_access'], 'rail': ['rail_only'], 'transit': ['bus_only', 'rail_only', 'drive_access'], 'walk': ['walk_access']})[source]

Determines if the network graph is “strongly” connected A graph is strongly connected if each vertex is reachable from every other vertex.

Parameters:

  • links_df – DataFrame of standard network links

  • nodes_df – DataFrame of standard network nodes

  • mode – mode of the network, one of drive,`transit`, walk, bike

  • modes_to_network_link_variables – dictionary mapping the mode selections to the network variables that must bool to true to select that mode. Defaults to MODES_TO_NETWORK_LINK_VARIABLES

Returns: networkx: osmnx: DiGraph of network

Returns nodes and link dataframes for specific mode.

Parameters:

  • links_df – DataFrame of standard network links

  • nodes_df – DataFrame of standard network nodes

  • modes – list of the modes of the network to be kept, must be in drive,`transit`,`rail`,`bus`, walk, bike. For example, if bike and walk are selected, both bike and walk links will be kept.

  • modes_to_network_link_variables – dictionary mapping the mode selections to the network variables that must bool to true to select that mode. Defaults to MODES_TO_NETWORK_LINK_VARIABLES

Returns: tuple of DataFrames for links, nodes filtered by mode

Todo

Right now we don’t filter the nodes because transit-only

links with walk access are not marked as having walk access Issue discussed in https://github.com/wsp-sag/network_wrangler/issues/145 modal_nodes_df = nodes_df[nodes_df[mode_node_variable] == 1]

get_property_by_time_period_and_group(prop, time_period=None, category=None, default_return=None)[source]

Return a series for the properties with a specific group or time period.

Parameters:

  • prop (str) – the variable that you want from network

  • time_period (list(str)) – the time period that you are querying for i.e. [‘16:00’, ‘19:00’]

  • category (str or list(str)(Optional)) –

    the group category i.e. “sov”

    or

    list of group categories in order of search, i.e. [“hov3”,”hov2”]

  • default_return (what to return if variable or time period not found. Default is None.) –

Return type:

pandas series

identify_segment(O_id, D_id, selection_dict={}, mode=None, nodes_df=None, links_df=None)[source]
Parameters:

  • endpoints – list of length of two unique keys of nodes making up endpoints of segment

  • selection_dict – dictionary of link variables to select candidate links from, otherwise will create a graph of ALL links which will be both a RAM hog and could result in odd shortest paths.

  • segment_variables – list of variables to keep

identify_segment_endpoints(mode='', links_df=None, nodes_df=None, min_connecting_links=10, min_distance=None, max_link_deviation=2)[source]
Parameters:

  • mode – list of modes of the network, one of drive,`transit`, walk, bike

  • links_df – if specified, will assess connectivity of this links list rather than self.links_df

  • nodes_df – if specified, will assess connectivity of this nodes list rather than self.nodes_df

is_network_connected(mode=None, links_df=None, nodes_df=None)[source]

Determines if the network graph is “strongly” connected A graph is strongly connected if each vertex is reachable from every other vertex.

Parameters:

  • mode – mode of the network, one of drive,`transit`, walk, bike

  • links_df – DataFrame of standard network links

  • nodes_df – DataFrame of standard network nodes

Returns: boolean

Todo

Consider caching graphs if they take a long time.

static load_transform_network(node_filename, link_filename, shape_filename, crs=4326, node_foreign_key='model_node_id', validate_schema=True, **kwargs)[source]

Reads roadway network files from disk and transforms them into GeoDataFrames.

Return type:

tuple

Parameters:

  • node_filename – file name for nodes.

  • link_filename – file name for links.

  • shape_filename – file name for shapes.

  • crs – coordinate reference system. Defaults to value in CRS.

  • node_foreign_key – variable linking the node table to the link table. Defaults to NODE_FOREIGN_KEY.

  • validate_schema – boolean indicating if network should be validated to schema.

returns: tuple of GeodataFrames nodes_df, links_df, shapes_df

static network_connection_plot(G, disconnected_subgraph_nodes)[source]

Plot a graph to check for network connection.

Parameters:

  • G – OSMNX flavored networkX graph.

  • disconnected_subgraph_nodes – List of disconnected subgraphs described by the list of their member nodes (as described by their model_node_id).

returns: fig, ax : tuple

orig_dest_nodes_foreign_key(selection, node_foreign_key='')[source]

Returns the foreign key id (whatever is used in the u and v variables in the links file) for the AB nodes as a tuple.

Return type:

tuple

Parameters:

  • selection – selection dictionary with A and B keys

  • node_foreign_key – variable name for whatever is used by the u and v variable

  • specified (in the links_df file. If nothing is) –

  • whatever (assume) –

  • is (default) –

Returns: tuple of (A_id, B_id)

static ox_graph(nodes_df, links_df, node_foreign_key='model_node_id', link_foreign_key=['A', 'B'], unique_link_key='model_link_id')[source]

create an osmnx-flavored network graph

osmnx doesn’t like values that are arrays, so remove the variables that have arrays. osmnx also requires that certain variables be filled in, so do that too.

Parameters:

  • nodes_df – GeoDataFrame of nodes

  • link_df – GeoDataFrame of links

  • node_foreign_key – field referenced in link_foreign_key

  • link_foreign_key – list of attributes that define the link start and end nodes to the node foreign key

  • unique_link_key – primary key for links

Returns: a networkx multidigraph

Parameters:

  • candidate_links – selection of links geodataframe with links likely to be part of path

  • O_id – origin node foreigh key ID

  • D_id – destination node foreigh key ID

  • weight_column – column to use for weight of shortest path. Defaults to “i” (iteration)

  • weight_factor – optional weight to multiply the weight column by when finding the shortest path

  • search_breadth –

Returns

static read(link_filename, node_filename, shape_filename, fast=True, crs=4326, node_foreign_key='model_node_id', link_foreign_key=['A', 'B'], shape_foreign_key='id', unique_link_key='model_link_id', unique_node_key='model_node_id', unique_link_ids=['model_link_id'], unique_node_ids=['model_node_id'], modes_to_network_link_variables={'bike': ['bike_access'], 'bus': ['bus_only', 'drive_access'], 'drive': ['drive_access'], 'rail': ['rail_only'], 'transit': ['bus_only', 'rail_only', 'drive_access'], 'walk': ['walk_access']}, modes_to_network_nodes_variables={'bike': ['bike_node'], 'bus': ['bus_only', 'drive_access'], 'drive': ['drive_access'], 'rail': ['rail_only', 'drive_access'], 'transit': ['bus_only', 'rail_only', 'drive_access'], 'walk': ['walk_node']}, managed_lanes_link_id_scalar=10000000, managed_lanes_node_id_scalar=4500000, managed_lanes_required_attributes=['A', 'B', 'model_link_id', 'locationReferences'], keep_same_attributes_ml_and_gp=['distance', 'bike_access', 'drive_access', 'transit_access', 'walk_access', 'maxspeed', 'name', 'oneway', 'ref', 'roadway', 'length', 'segment_id', 'ft', 'assignable', 'county'])[source]

Reads a network from the roadway network standard Validates that it conforms to the schema

Return type:

RoadwayNetwork

Parameters:

  • node_filename – full path to the node file

  • link_filename – full path to the link file

  • shape_filename – full path to the shape file

  • fast – boolean that will skip validation to speed up read time

  • crs – coordinate reference system, ESPG number

  • node_foreign_key – variable linking the node table to the link table.

  • link_foreign_key –

  • shape_foreign_key –

  • unique_link_ids –

  • unique_node_ids –

  • modes_to_network_link_variables –

  • modes_to_network_nodes_variables –

  • managed_lanes_node_id_scalar –

  • managed_lanes_link_id_scalar –

  • managed_lanes_required_attributes –

  • keep_same_attributes_ml_and_gp –

Returns: a RoadwayNetwork instance

Todo

Turn off fast=True as default

static roadway_net_to_gdf(roadway_net)[source]
Return type:

GeoDataFrame

Turn the roadway network into a GeoDataFrame :param roadway_net: the roadway network to export

returns: shapes dataframe

Todo

Make this much more sophisticated, for example attach link info to shapes

select_roadway_features(selection, search_mode='drive', force_search=False, sp_weight_factor=None)[source]

Selects roadway features that satisfy selection criteria

Return type:

GeoDataFrame

Example usage:
net.select_roadway_features(
selection = [ {

# a match condition for the from node using osm, # shared streets, or model node number ‘from’: {‘osm_model_link_id’: ‘1234’}, # a match for the to-node.. ‘to’: {‘shstid’: ‘4321’}, # a regex or match for facility condition # could be # of lanes, facility type, etc. ‘facility’: {‘name’:’Main St’}, }, … ])

Parameters:

  • selection – dictionary with keys for: A - from node B - to node link - which includes at least a variable for name

  • search_mode – mode which you are searching for; defaults to “drive”

  • force_search – boolean directing method to perform search even if one with same selection dict is stored from a previous search.

  • sp_weight_factor – multiple used to discourage shortest paths which meander from original search returned from name or ref query. If not set here, will default to value of sp_weight_factor in RoadwayNetwork instance. If not set there, will defaul to SP_WEIGHT_FACTOR.

Returns: a list of link IDs in selection

Return type:

bool

Parameters:

selection_dictionary – Dictionary representation of selection of roadway features, containing a “link” key.

Returns: A boolean indicating if the selection dictionary contains

a unique identifier for links.

selection_map(selected_link_idx, A=None, B=None, candidate_link_idx=[])[source]

Shows which links are selected for roadway property change or parallel managed lanes category of roadway projects.

Parameters:

  • selected_links_idx – list of selected link indices

  • candidate_links_idx – optional list of candidate link indices to also include in map

  • A – optional foreign key of starting node of a route selection

  • B – optional foreign key of ending node of a route selection

shortest_path(graph_links_df, O_id, D_id, nodes_df=None, weight_column='i', weight_factor=100)[source]
Return type:

tuple

Parameters:

  • graph_links_df –

  • O_id – foreign key for start node

  • D_id – foreign key for end node

  • nodes_df – optional nodes df, otherwise will use network instance

  • weight_column – column to use as a weight, defaults to “i”

  • weight_factor – any additional weighting to multiply the weight column by, defaults to SP_WEIGHT_FACTOR

Returns: tuple with length of four - Boolean if shortest path found - nx Directed graph of graph links - route of shortest path nodes as List - links in shortest path selected from links_df

update_distance(links_df=None, use_shapes=False, units='miles', network_variable='distance', overwrite=True, inplace=True)[source]

Calculate link distance in specified units to network variable using either straight line distance or (if specified) shape distance if available.

Parameters:

  • links_df – Links GeoDataFrame. Useful if want to update a portion of network links (i.e. only centroid connectors). If not provided, will use entire self.links_df.

  • use_shapes – if True, will add length information from self.shapes_df rather than crow-fly. If no corresponding shape found in self.shapes_df, will default to crow-fly.

  • units – units to use. Defaults to the standard unit of miles. Available units: “meters”, “miles”.

  • network_variable – variable to store link distance in. Defaults to “distance”.

  • overwrite – Defaults to True and will overwrite all existing calculated distances. False will only update NaNs.

  • inplace – updates self.links_df

Returns:

links_df with updated distance

Validate roadway network data link schema and output a boolean

static validate_node_schema(node_file, schema_location='roadway_network_node.json')[source]

Validate roadway network data node schema and output a boolean

validate_properties(properties, ignore_existing=False, require_existing_for_change=False)[source]

If there are change or existing commands, make sure that that property exists in the network.

Return type:

bool

Parameters:

  • properties – properties dictionary to be evaluated

  • ignore_existing – If True, will only warn about properties that specify an “existing” value. If False, will fail.

  • require_existing_for_change – If True, will fail if there isn’t a specified value in theproject card for existing when a change is specified.

Returns: boolean value as to whether the properties dictonary is valid.

validate_selection(selection, selection_requires=['link'])[source]

Evaluate whetther the selection dictionary contains the minimum required values.

Return type:

bool

Parameters:

selection – selection dictionary to be evaluated

Returns: boolean value as to whether the selection dictonary is valid.

static validate_shape_schema(shape_file, schema_location='roadway_network_shape.json')[source]

Validate roadway network data shape schema and output a boolean

validate_uniqueness()[source]

Confirms that the unique identifiers are met.

Return type:

bool

write(path='.', filename=None)[source]

Writes a network in the roadway network standard

Return type:

None

Parameters:

  • path – the path were the output will be saved

  • filename – the name prefix of the roadway files that will be generated