The Victorian maritime threat and error management project - Scientific report on the development and validation of data collection tools

Abstract

The first stage of this project has aimed to develop a scientifically valid and reliable tool for the qualitative assessment of ships‟ crew performance while under pilotage. While many of the deficiencies in crew performance are well-known to those in the industry, it is hoped that a credible scientific assessment will provide the funding organizations with valuable information to advocate for improvements in a range of areas such as maritime education and training, port and pilotage risk assessment and maritime regulations. Qualitative and quantitative methods were employed to determine whether a valid description of bridge crew behavior could be developed and utilized in safety audit processes. Five expert sea pilots participated in a Hierarchical Task Analysis procedure that identified that tasks of safe navigation through pilotage waters and associated crew behaviours. Another five experienced seafarers participated in the recording of a simulation exercise that demonstrated varying levels of crew performance. A total of 10 experts then rated the behavioural makers. Inter-rater agreement was analysed and the overall mean rWG values increased from .92 to .98 following a Delphi procedure. These findings suggest that the identified behavioural markers constitute highly reliable descriptions of safe crew performance. The tool is therefore ready to be taken into the field and used for data collection.