National Technical Reports Library

The National Technical Information Service acquires, indexes, abstracts, and archives the largest collection of U.S. government-sponsored technical reports in existence. The NTRL offers online, free and open access to these authenticated government technical reports. Technical reports and documents in its repository may be available online for free either from the issuing federal agency, the U.S. Government Publishing Office’s Federal Digital System website, or through search engines.

Details

Use of Mahalanobis Distance for Detecting Outliers and Outlier Clusters in Markedly Non-Normal Data: A Vehicular Traffic Example.

ADA545834

Publication Date	2011
Personal Author	Warren, R.; Smith, R. F.; Cybenko, A. K.
Page Count	60
Abstract	Modeling the behavior of interacting humans in routine but complex activities has many challenges, not the least of which is that humans can be both purposive and negligent, and further can encounter unexpected environmental hazards requiring fast action. The challenge is to characterize and model the humdrum routine while at the same time capturing the deviations and anomalies which arise from time to time. Because of the disruptive impact that anomalies (such as accidents) can have and the importance for incorporating them in our models, this report focuses on one technique for identifying anomalies in complex behavior patterns especially when there is no sharp demarcation between routine and unusual activity. The technique we evaluate is that of Mahalanobis distance which is known to be useful for identifying outliers when data is multivariate normal. But, the data we use for evaluation is deliberately markedly non-multivariate normal since that is what we confront in complex human systems. Specifically, we use one year's (2008) hourly traffic-volume data on a major multi-lane road (I-95) in one location in a major city (New York) with a dense population and several alternate routes. The traffic data is rich, large, incomplete, and reflects the effects of bad weather, accidents, routine fluctuations (rush hours versus dead of night), and onetime social events. The results show that Mahalanobis distance is a useful technique for identifying both single-hour outliers and contiguous-time clusters whose component members are not, in themselves, highly deviant.
Keywords	Abnormalities Traffic Behavior Interactions Humans Population Variations High density Patterns Routing Vehicles New york Hazards Weather Accidents Position(Location) Environments Mahalanobis distance
Source Agency	Non Paid ADAS
NTIS Subject Category	92B - Psychology 57T - Psychiatry 72F - Statistical Analysis
Corporate Authors	SRA International, Inc., Dayton, OH.
Supplemental Notes	Prepared in cooperation with Univ of Dayton Research Center, OH.
Document Type	Technical Report
Title Note	Interim rept. Mar 2009-Jun 2011.
NTIS Issue Number	201124
Contract Number	FA8650-09-D-6939-TO0023

Use of Mahalanobis Distance for Detecting Outliers and Outlier Clusters in Markedly Non-Normal Data: A Vehicular Traffic Example.

Use of Mahalanobis Distance for Detecting Outliers and Outlier Clusters in Markedly Non-Normal Data: A Vehicular Traffic Example.
ADA545834

Publication Date	2011
Personal Author	Warren, R.; Smith, R. F.; Cybenko, A. K.
Page Count	60
Abstract	Modeling the behavior of interacting humans in routine but complex activities has many challenges, not the least of which is that humans can be both purposive and negligent, and further can encounter unexpected environmental hazards requiring fast action. The challenge is to characterize and model the humdrum routine while at the same time capturing the deviations and anomalies which arise from time to time. Because of the disruptive impact that anomalies (such as accidents) can have and the importance for incorporating them in our models, this report focuses on one technique for identifying anomalies in complex behavior patterns especially when there is no sharp demarcation between routine and unusual activity. The technique we evaluate is that of Mahalanobis distance which is known to be useful for identifying outliers when data is multivariate normal. But, the data we use for evaluation is deliberately markedly non-multivariate normal since that is what we confront in complex human systems. Specifically, we use one year's (2008) hourly traffic-volume data on a major multi-lane road (I-95) in one location in a major city (New York) with a dense population and several alternate routes. The traffic data is rich, large, incomplete, and reflects the effects of bad weather, accidents, routine fluctuations (rush hours versus dead of night), and onetime social events. The results show that Mahalanobis distance is a useful technique for identifying both single-hour outliers and contiguous-time clusters whose component members are not, in themselves, highly deviant.
Keywords	Abnormalities Traffic Behavior Interactions Humans Population Variations High density Patterns Routing Vehicles New york Hazards Weather Accidents Position(Location) Environments Mahalanobis distance
Source Agency	Non Paid ADAS
NTIS Subject Category	92B - Psychology 57T - Psychiatry 72F - Statistical Analysis
Corporate Authors	SRA International, Inc., Dayton, OH.
Supplemental Notes	Prepared in cooperation with Univ of Dayton Research Center, OH.
Document Type	Technical Report
Title Note	Interim rept. Mar 2009-Jun 2011.
NTIS Issue Number	201124
Contract Number	FA8650-09-D-6939-TO0023