Improving the Automatic Classification of Interviewer Questions in Avatar‐Based Child Investigative Interview Simulations: A Transformer Approach

Abstract

Purpose

Interviewer questioning style, including the types of questions asked, represents one measurable and trainable component of child investigative interview quality. Although automated approaches to question classification have been explored, previous systems have shown only moderate performance, particularly when distinguishing fine-grained categories of questions. The present study aimed to improve the automatic multi-class classification of interviewer questions and the detection of repetitive questioning patterns in avatar-based simulations of child sexual abuse investigative interviews using transformer-based architectures.

Method

The system was trained on 17,425 annotated questions derived from controlled avatar-based interview simulations and categorized into 14 question-category labels. A transformer model (F2LLM-0.6B) was fine-tuned for multi-class classification using weighted loss to address class imbalance. A separate LLM-based module (Qwen3.5-9B) detected semantically repeated or reformulated questions within interview sequences. Performance was evaluated using accuracy, macro- and weighted F1-scores, and confusion matrix analyses.

On a held-out test set (N = 2, 614), the classification model achieved an overall accuracy of 86.8% and a macro-F1 score of 79.0%, while the repetition-detection module achieved an accuracy of 84.9%.

Conclusions

These findings indicate that AI systems can approximate the available human-coded reference labels for interviewer question types and identify repeated or reformulated questioning patterns. The system supports taxonomy-based coding of interviewer questions; it does not independently determine their pragmatic or contextual appropriateness within the broader interview sequence, nor does it assess overall interview quality. The proposed approach provides a scalable framework for automated question-type coding in avatar-based training and research settings while preserving the central role of professional judgement.

Go to Source