Teaching AI-Integrated Engineering Through Project-Based Learning: A Case Study on Semi-Supervised Representation Learning for Discrete Audio Unit Classification

Abstract

This paper presents a project-based learning (PBL) approach to teaching AI-integrated engineering through a case study on semi-supervised representation learning for dis-create audio unit (phoneme) classification. The proposed system, speech2phone, combines manifold learning via Uniform Manifold Approximation and Projection (UMAP) with consistency-based semi-supervised learning algorithms such as the Pi-Model and Label Gradient Alignment (LGA). By leveraging a small amount of labeled data alongside a large corpus of unlabeled speech, the method addresses the fundamental challenge of data scarcity in low-resource language processing. Experiments conducted on the TIMIT acoustic-phonetic corpus demonstrate that UMAP embeddings reduce the Phoneme Error Rate (PER) to 23.1% when used as a preprocessing step for a 1D CNN, outper-forming PCA, t-SNE, and variational autoencoders. In semi-supervised settings using only 10% labeled data, the UMAP+Pi-Model achieves a PER of 28.5%, significantly outperforming the supervised CNN baseline (31.2% PER). With full labeled data, the method attains 18.4% PER, surpassing the supervised BiLSTM baseline (22.3% PER). An ablation study confirms the complementary benefits of UMAP and semi-supervised regular-ization. The framework requires only 3 hours of training on an NVIDIA RTX 3080 and achieves 12 ms inference time per audio second, making it suitable for real-time applications. Ethical considerations regarding speaker privacy are also discussed. This case study illustrates how PBL can effectively integrate representation learning, semi-supervised methods, and speech processing into an AI engineering curriculum.