Agent for CUBE Demo: Technical Note


Download the proof-of-concept (demo) here: 

communities.bentley.com/.../ex_5F00_Winnipeg_5F00_Agent_5F00_20230106_5F00_1657.zip 

For help using the proof-of-concept example, please contact: filippo.contiero@bentley.com

Introduction

Motivation

The motivation behind the integration of Agent within CUBE Application Manager is based on the following main objectives:

Agent might be used to re-platform an existing travel demand model, whether or not it is implemented directly in Voyager, or to develop entirely new model structures to support transport planning decisions.

When integrating Agent within CUBE, data processing and travel model components as well as organization of data and scenarios may continue to use the CUBE Application Manager and Scenario Manager, which can manage the integration between Agent scenarios.

About Agent

Agent provides a platform to assemble, calibrate and apply agent-based travel demand models, that is, demand models which are expressed entirely as a choice of individual agents including persons, households, trips, tours and even aggregate O-D pairs or zones. Agent supports a variety of travel demand model structures and may be used to implement trip-based, hybrid, tour-based or activity-based models, as well as to mix elements of each as needed.

More background information about Agent is available at the following links:

Proof-of-Concept

This proof-of-concept has been developed to demonstrate the feasibility of this integration and may be generalized to allow for more complex model structures and extended to integrate the usage of CUBE Land for generating demographic data for Agent. For detailed technical appreciation of the how the integration is accomplished, please see Appendix – Integration and API details. To use the proof-of-concept model, please follow the step-by-step explanation below (do not skip any step!).

AGENT Add-on: Licensing and Distribution

AGENT is available to all CUBE users via the AGENT Add-on. At time of publishing (Nov 2022), the AGENT Add-on is distributed and documented via EMME software which must be downloaded and installed prior to use. It may be useful for the reader to consult additional references listed below (see "Further Reading").

Agent for CUBE Winnipeg Demo: Key Concepts

Agent data is stored into the Datastore, stored in a separate and independent database in the directory Datastore.

Agent demographic scenarios is based on different Land Use Data, grouping together:

Agent Model Packages represent an assembled configuration of a certain type of a demand model, in the case of the Winnipeg demo, this is a disaggregate tour-based model. This model package also includes a list of targets which can be used to calibrate the demand model.

Agent Travel Scenario is defined through the Python scripts, using the API, just as a “container” where the demand model package is run based on the selection of:

The user should notice that the implementation for this proof-of-concept can be further generalized to become more flexible and accommodate the full specification from the CUBE model for other model structures. In this example, the following two Python files are provided and they should be customized by the user if other model structures related to the user specific models need to be applied:

Finally, to open the Agent GUI natively from CUBE Application Manager, the user can run the following pyc file from CUBE:

**start "" "{Python_Interpreter}" "{catalog_dir}/AgentPy/open_agent.cpython-37.pyc" "{catalog_dir}"

Agent for CUBE Winnipeg Demo: Users Workflow

After running the ex_Winnipeg_Agent_v0103.exe to unzip the content, the Agent for CUBE Winnipeg Demo catalog is provided in the following location:

"…\ex_Winnipeg_Agent\ex_Winnipeg_Agent_v0103.cat"

The CUBE catalog folder structure looks initially like the below:

To use the Agent for CUBE Proof-of-Concept catalog you should open the CUBE CONNECT Edition (v6.5) software and open the catalog ex_Winnipeg_Agent.cat.

There are two main (parent) applications within the catalog:

To open the help for Agent, look at the “Agent_Help” application and double-click on the Pilot program corresponding to the manual you would like to open.

Then, to work with the proof-of-concept model for Winnipeg, you should open (double-click) on the ex_Demo_Winnipeg application (see screenshot above).

You can start by using the existing Scenario_3001, and then you can also define your own scenario and select your inputs. Nevertheless, it is important to notice that the {Scenario_Code} that is defined in CUBE, corresponds to the Emme emmebank scenario id for the network scenario in the Emme project that will be generated. Therefore a single integer value should be entered here, between 1 and 99999.

When using this demo for the first time, please make sure to point the Python interpreter key to the EMME installation folder for the version installed on your machine, as shown in the below screenshot as example:

Before running the entire demo model scenario, it is necessary to run the application in "set-up mode", i.e., to only set-up the inputs for Agent model preparation and to open the Agent GUI through CUBE, with the populated inputs. This can be done by selecting the option within the CUBE Scenario Panel for the scenario, checking the box below:

Alternatively, and probably more commonly when building a model from scratch and checking each step, it is possible to run the CUBE application step-by-step, to create the necessary initialization inputs for the Agent model preparation. Therefore, it is alternatively possible to run the following sub-groups through Application Manager:

Then, once those intermediate files are created, run (double-click) on the Pilot program inside the “Demand model (Agent)” sub-group, with name “Create/Edit Demographic Scenario”, to create (if not already run) or update (if this has been already run successfully) the Emme project with network scenario, in the background, and the Agent model with demographic scenario, through the Python script using the API. You will notice that this and the next Pilot program need to be run manually only when needed during the model development (they can even be removed or hidden in the final version of the model, once the Agent model has been created and does not need to be updated, to be used by the model applier if desired).

Once the Agent model has been created, or updated with the CUBE inputs, the Python pop-up window will close. If you look at your catalog directory, the first time that you run this Pilot program you will notice that the Emme project folder below will be created and populated:

Then, run (double-click) the Pilot program named “Open Agent” within the same sub-group, this will open the Agent GUI for you to build your model with the already populated inputs from the previous steps.

When you open Agent through CUBE, you will see a window like the one below opening up, showing you the arguments that you passed to the Python script that are initializing your Agent model. The Agent window will be opened behind that window. Do not close the Python window when working in the Agent window.

Once the Agent GUI opens up, with your network and demographic scenarios already populated through the previous steps from your set of inputs coming from CUBE, then you can proceed with building your Agent model through the interface.

To make things quicker for the purpose of the demo, we have prepared some specification files to be used directly, avoiding having you to re-build the entire model:

Note: the name of this Travel Scenario is specified in the specification file created within CUBE. If a different name is to be used, this needs to be updated in CUBE under the catalog key (scenario specific): {model_package}

Once the Population Synthesis and the Model Package have been created, close the Agent GUI (click "Confirm" when asked to close the GUI).

From this point on, the Travel Scenarios will be created for any new scenario directly through CUBE, without the need to open Agent again.

If double-clicking again the same Pilot step mentioned above to open Agent, this will just open the Agent GUI again but with the existing specifications with the most updated data from CUBE, it will not re-create the project.

Once the above steps are undertaken and the Agent window has been closed, it is possible to run the entire Scenario_3001 by double clicking on it within CUBE Scenario Manager and click “run”, making sure to uncheck the box selecting the "set-up mode". 

This will run the CUBE feedback loop, with the Agent demand model fully integrated. You will see the Agent window temporarily opening up during the model execution, do not close the window, it will close automatically.

The full model should take around 8 minutes to run 2 global loops, tested in a Laptop with the following characteristics:

Disclaimer: Current Workflow Limitations

At the current state of development, the provided AGENT for CUBE demo (proof-of-concept) tool requires the installation of both CUBE and EMME, and may not provide clear user feedback in the following cases:

In these cases, it is always recommended to first start AGENT from the corresponding EMME project (.emp) in EMME Desktop to receive full information on required data upgrades, backup and other messages. More details may be found in EMME Help->Agent Manual->Launch Agent.

Further Reading

The reader interested in additional references may consult the following resources available in the EMME distribution:

Appendix: Integration and API Details

The integration of Agent in CUBE is currently implemented via the EMME and Agent APIs in Python scripts that may be run in CUBE within Pilot steps. Data are passed from CUBE to Agent through system arguments, and through catalog keys for the Agent specification file, which may also be parametrized.

The proof-of-concept illustrates the use of the abovementioned Emme and Agent APIs to automate the construction of a minimal Emme project and Agent datastore based on CUBE model data. The following steps are currently required to guarantee consistency between the CUBE model and the Agent model set-up.

1. Inputs:
a. Creation of an Emme project (*.emp file) and folder within the CUBE catalog folder
b. For input skim matrices we convert *.mat CUBE matrices to OMX matrices which are then imported into the Emme project.
c. For zonal data, we import CSV files into the Agent datastore via the Agent API.
d. For centroids, we import a list of TAZ-level centroids as CSV into the Emme database. The centroid IDs here must correspond to the corresponding Agent zone system or validation errors will be raised.
e. The travel scenario configuration file is in json format and is parametrized to use catalog keys through a Voyager Matrix program. The extension of the configuration file is *.ems.

2. Outputs:
a. Demand model simulation results including aggregate (O-D, zonal) results, or disaggregate (households, persons, trips, tours, etc.) results are stored in a corresponding Agent travel scenario.
b. Trip table matrices are written to the Emme Database. These are then converted to OMX using Emme APIs, and then converted to CUBE *.mat format using a Pilot step with CONVERTMAT.

The user should be aware of the following three aspects specifically related to the Agent API: