USC SAIL · NSTC Fellow

Huang-Cheng Chou

周惶振

Speech / multimodal researcher for voice assistants— speech emotion recognition, subjective evaluation, and systems that hear affect as carefully as words.

Huang-Cheng Chou (周惶振)

About

I am a Postdoctoral Scholar–NSTC Fellow at the Signal Analysis and Interpretation Laboratory (SAIL) at the University of Southern California, working with Prof. Shrikanth S. Narayanan. I received my Ph.D. in Electrical Engineering from National Tsing Hua University, advised by Prof. Chi-Chun Lee.

My research centers on three threads: (1) speech emotion recognition under subjectivity and ambiguity— multi-label learning, annotator disagreement, calibration, fairness, and open evaluation via EMO-SUPERB; (2) conversational social signal processing—deception and belongingness in dialogue / small groups, and dyadic clinical affect such as depression-related speech; and (3) speech systems for assistants and multilingual / low-resource settings—unified ASR + SER, speech LLMs, expressive TTS evaluation, edge SER / KWS, and collaborations such as TaigiSpeech.

I have worked on speech and ML systems from research through deployment at Amazon Alexa, ITRI, and RealTek. I am motivated by building intelligent, expressive agents that integrate speech, language, and emotion for natural interaction.

Current
Postdoctoral Scholar (NSTC Fellow), USC SAIL / SPAN, Oct 2025 – Present
Education
Ph.D., Electrical Engineering, NTHU, 2024; B.S., Electrical Engineering, NTHU
Contact
huangchengchou@gmail.com

News

  1. Contributing to USC SAIL SPAN (Speech Production and Articulation Knowledge): preparing a first-author manuscript on speech enhancement for real-time MRI (rtMRI) speech —multi-task benchmark and open resource to improve usability of SPAN rtMRI audio (targeting journal submission by end of month; PDF after preprint / acceptance).
  2. Updating VoxEmo, a reproducibility toolkit for speech-LLM SER (mentored; arXiv).
  3. Started as an NSTC Postdoctoral Fellow at USC SAIL with Prof. Shrikanth Narayanan.
  4. Our paper TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset… (coauthor) is available on arXiv (INTERSPEECH 2026 long paper).
  5. DeSTA2.5-Audio is published in IEEE TASLP 2026.
  6. Do You Hear What I Mean? (instruction–perception gap in expressive TTS) is accepted by ICASSP 2026.
  7. First-author work on stimulus modality for SER at ICASSP 2025; INTERSPEECH 2025 papers on SER fairness and Meta-PerSER.
  8. Co-led EMO-SUPERB / Open-Emotion (IEEE SLT 2024); Minority Views Matter in IEEE TAFFC.
  9. Received the NSTC Postdoctoral Research Abroad Fellowship (2025–2026).
  10. Ph.D. dissertation on SER under subjectivity received the ACLCLP Doctoral Dissertation Award — Honorable Mention.

Projects

Grouped by research theme. Selected papers under each; full list on Scholar and in the CV.

Theme

Conversational social signals

Conversational deception detection pipeline
Conversational deception · APSIPA ASC 2019

Affect and social meaning in dialogue: deception and perceived lying in interactive games, belongingness / satisfaction in small-group conversation, multimodal group emotion, and dyadic depression-related speech (including mentored work).

Theme

Fairness & robust SER

Subgroup disparity, debiasing without fragile demographic shortcuts, and confidence-oriented augmentation for fairer multi-label emotion recognition.

  • Emo-bias — INTERSPEECH 2024
  • Mitigating subgroup disparities — INTERSPEECH 2025
  • EMO-Debias — ASRU 2025
  • CO-VADA — ASRU 2025

Open evaluation

Benchmarks & toolkits

EMO-SUPERB community evaluation loop
EMO-SUPERB · IEEE SLT 2024

Reproducible evaluation contracts for classical and speech-LLM SER: splits, metrics, prompts, parsing, and diagnostics so results stay auditable.

Theme

Assistant speech systems

Tiny Whisper-SER multitask architecture
Tiny Whisper-SER · APSIPA ASC 2024

Unified ASR + multi-label SER for voice-assistant workloads, emotional-speech ASR analysis, stimulus-modality effects, and on-device SER / KWS under compute budgets.

  • A Tiny Whisper-SER — APSIPA ASC 2024
  • Amazon Alexa Applied Scientist Intern — shared-layer ASR + SER on Whisper
  • Stimulus modality matters — ICASSP 2025
  • RealTek internship — streaming edge SER / KWS compression

Theme

Speech LLMs, generation & low-resource

DeSTA2.5 self-generated alignment training pipeline
DeSTA2.5-Audio · IEEE TASLP 2026

Large audio / speech language models, expressive TTS evaluation, codec emotion preservation, and low-resource speech resources such as TaigiSpeech.

USC SAIL · SPAN

rtMRI speech enhancement for SPAN

Contribution to SAIL’s Speech Production and Articulation Knowledge (SPAN) effort on real-time MRI of dynamic vocal-tract shaping. First-author work evaluates modern speech enhancement on rtMRI speech—objective quality plus downstream tasks (ASR, speaker, SER, demographics)— and releases practical guidance with an open resource so SPAN corpora are more usable beyond speech science. Manuscript in preparation (JASA-style venue); PDF will be posted after preprint or acceptance.

Selected publications

Full list on Google Scholar and in the CV appendix. Name in bold indicates Huang-Cheng Chou.

Selected first-author

  1. Minority Views Matter: Evaluating Speech Emotion Classifiers with Human Subjective Annotations by an All-Inclusive Aggregation Rule Huang-Cheng Chou, Lucas Goncalves, Seong-Gyun Leem, Ali N. Salman, Chi-Chun Lee, Carlos Busso IEEE Transactions on Affective Computing, 2024 DOI
  2. Open-Emotion: A Reproducible EMO-Superb For Speech Emotion Recognition Systems Huang-Cheng Chou, Haibin Wu, Kai-Wei Chang, Lucas Goncalves, Jiawei Du, Jyh-Shing Roger Jang, Chi-Chun Lee, Hung-Yi Lee IEEE SLT 2024 DOI · Project
  3. Embracing Ambiguity And Subjectivity Using The All-Inclusive Aggregation Rule For Evaluating Multi-Label Speech Emotion Recognition Systems Huang-Cheng Chou, Lucas Goncalves, Haibin Wu, Hung-Yi Lee, Chi-Chun Lee IEEE SLT 2024 DOI
  4. A Tiny Whisper-SER: Unifying Automatic Speech Recognition and Multi-label Speech Emotion Recognition Tasks Huang-Cheng Chou APSIPA ASC 2024 DOI
  5. Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance Huang-Cheng Chou, Haibin Wu, Chi-Chun Lee ICASSP 2025 DOI
  6. The Importance of Calibration: Rethinking Confidence and Performance of Speech Multi-label Emotion Classifiers Huang-Cheng Chou, Lucas Goncalves, Seong-Gyun Leem, Chi-Chun Lee, Carlos Busso INTERSPEECH 2023 DOI
  7. Every Rating Matters: Joint Learning of Subjective Labels and Individual Annotators for Speech Emotion Classification Huang-Cheng Chou, Chi-Chun Lee ICASSP 2019 DOI

Selected collaborative

  1. DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model With Self-Generated Cross-Modal Alignment Ke-Han Lu, …, Huang-Cheng Chou, …, Hung-yi Lee IEEE TASLP 2026 DOI
  2. TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild Kai-Wei Chang, Yi-Cheng Lin, Huang-Cheng Chou, …, Hung-yi Lee INTERSPEECH 2026 (long), arXiv arXiv
  3. Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-to-Speech Systems Yi-Cheng Lin, Huang-Cheng Chou, Tzu-Chieh Wei, Kuan-Yu Chen, Hung-Yi Lee ICASSP 2026 DOI
  4. VoxEmo: A Reproducible Toolkit for Speech Emotion Recognition with Speech LLMs Hezhao Zhang, Huang-Cheng Chou, Shrikanth Narayanan, Thomas Hain arXiv (mentored) arXiv
See all on Google Scholar →

Experience

  • 2025 – Present
    Postdoctoral Scholar (NSTC Fellow) USC SAIL · Prof. Shrikanth Narayanan
  • 2024 – 2025
    Independent Researcher Collab. with NTU SPML Lab (Prof. Hung-yi Lee), part-time during mandatory service
  • 2024
    Applied ML / DSP Intern RealTek · Emerging Tech / Advanced DSP
  • 2023
    Applied Scientist Intern Amazon AGI · Alexa Speech / ASR
  • 2016 – 2024
    Ph.D., Electrical Engineering NTHU BIIC Lab · Prof. Chi-Chun Lee

Selected awards

  • NSTC Postdoctoral Research Abroad Fellowship — 2025–2026
  • ACLCLP Doctoral Dissertation Award — Honorable Mention (2024)
  • Merry Electronics Electroacoustics Thesis Award — Silver (2025), Bronze (2021)
  • APSIPA ASC Best Regular Paper Award — 2019
  • NOVATEK Ph.D. Excellence Scholarship — 2022–2023
  • IEEE SPS ICASSP Travel Grant — 2025
  • IEEE SLT Travel Grant — 2024
  • Google East Asia Student Travel Grants — ICASSP 2022 & INTERSPEECH 2022
  • ISCA INTERSPEECH Grant — 2022
  • AAAC ACII Student Travel Grant — 2017
  • ACLCLP Outstanding Students Conference Travel Grant — 2019, 2022, 2024, 2025
  • FAOS Outstanding Students Conference Travel Grant — 2019, 2022, 2023

Additional scholarships and reviewing service are listed in the CV.

Contact

Open to conversations on speech AI, voice assistants, and research collaboration.

huangchengchou@gmail.com