Guidelines for Using StreetLight Data for Planning Tasks

About the report:

The Virginia Department of Transportation (VDOT) has purchased a subscription to the StreetLight (SL) Data products that mainly offer origin-destination (OD) related metrics through crowdsourcing data. Users can manipulate a data source like this to quickly estimate origin-destination trip tables. Nonetheless, the SL metrics heavily rely on the data points sampled from smartphone applications and global positioning services (GPS) devices, which may be subject to potential bias and coverage issues. In particular, the quality of the SL metrics in relation to meeting the needs of various VDOT work tasks is not clear. Guidelines on the use of the SL metrics are of interest to VDOT.

This study aimed to help VDOT understand the performance of the SL metrics in different application contexts. Specifically, existing studies that examined the potential of SL metrics have been reviewed and summarized. In addition, the experiences, comments, and concerns of existing users and potential users have been collected through online surveys. The developed surveys were primarily distributed to VDOT engineers and planners as well as other professionals in planning organizations and consultants in Virginia. Their typical applications of the SL metrics have been identified and feedback has been used to guide and inform the design of the guidelines.

To support the development of a set of guidelines, the quality of the SL metrics has been independently evaluated with six testing scenarios covering annual average daily traffic (AADT), origin-destination trips, traffic flow on road links, turning movements at intersections, and truck traffic. The research team has sought ground-truth data from different sources such as continuous count stations, toll transaction data, VDOT’s internal traffic estimations, etc. Several methods were used to perform the comparison between the benchmark data and the corresponding SL metrics. The evaluation results were mixed. The latest SL AADT estimates showed relatively small absolute percentage errors, whereas using the SL metrics to estimate OD trips, traffic counts on roadway segments and at intersections, and truck traffic did not show a relatively low and stable error rate. Large percentage errors were often found to be associated with lower volume levels estimated based on the SL metrics. In addition, using the SL metrics from individual periods as the input for estimating these traffic measures resulted in larger errors. Instead, the aggregation of data from multi-periods helped reduce the errors, especially for low volume conditions.  Depending on project purposes, the aggregation can be based on metrics of multiple days, weeks, or months.

The results from the literature review, surveys, and independent evaluations were synthesized to help develop the guidelines for using SL data products. The guidelines focused on five main aspects: (1) a summary for using SL data for typical planning work tasks; (2) general guidance for data extraction and preparation; (3) using the SL metrics in typical application scenarios; (4) quality issues and calibration of the SL metrics; and (5) techniques and tools for working with the SL metrics. The developed guidelines were accompanied with illustrative examples to allow users to go through the given use cases.

Based on the results, the study recommends that VDOT’s Transportation and Mobility Planning Division (TMPD) should encourage and support the use of the guidelines in projects involving SL data, and that TMPD should adopt a checklist (table) for reporting performance, calibration efforts, and benchmark data involved in projects that use the SL metrics.

Authors