This work is focused on the automatic recognition of environmental noise sources that affect humans’ health and quality of life, namely industrial, aircraft, railway and road traffic. However, the recognition of the latter, which have the largest influence on citizens’ daily lives, is still an open issue. Therefore, although considering all the aforementioned noise sources, this paper especially focuses on improving the recognition of road noise events by taking advantage of the perceived noise differences along the road vehicle pass-by (which may be divided into different phases: approaching, passing and receding). To that effect, a hierarchical classification scheme that considers these phases independently has been implemented. The proposed classification scheme yields an averaged classification accuracy of 92.5%, which is, in absolute terms, 3% higher than the baseline (a traditional flat classification scheme without hierarchical structure). In particular, it outperforms the baseline in the classification of light and heavy vehicles, yielding a classification accuracy 7% and 4% higher, respectively. Finally, listening tests are performed to compare the system performance with human recognition ability. The results reveal that, although an expert human listener can achieve higher recognition accuracy than the proposed system, the latter outperforms the non-trained listener in 10% in average.