Evaluating the Effectiveness of Machine Translation Tools in Translating China-specific Discourse

Liwei Zou

doi:10.54691/30qy6287

Evaluating the Effectiveness of Machine Translation Tools in Translating China-specific Discourse

Liwei Zou

Scientific Journal of Humanities and Social Sciences, (2025), Vol.7, No.9, pp.52-59

Published: September 9, 2025

DOI: 10.54691/30qy6287

PDF: Download Full Text PDF

Abstract

The accuracy of China-specific discourse translation has emerged as a critical factor in conveying China’s voice and shaping its global image with China’s comprehensive national strength steadily rising and international exchanges becoming increasingly frequent. This study conducts a comparative analysis between the official English translations of ten representative Chinese sentences from a selected China-specific discourse and the outputs of four mainstream international and domestic machine translation tools: Google Translate, ChatGPT, Wenxin Yiyan (Ernie Bot), and DeepSeek. The objective is to assess the effectiveness of these machine translation tools in the context of China-specific document translation. The evaluation combines automatic metrics—BLEU and TER—to quantify translation quality, and is further supplemented by human judgment to offer a more comprehensive understanding of each system’s performance. The results indicate that ChatGPT is the most robust overall, excelling in both machine and human metrics. ERNIE Bot shows strong potential in human-preferred translation of idiomatic and China-specific language, despite poor automatic scores. Finally, this study explores future trends in translation development in the era of artificial intelligence.

Keywords: Machine translation, China-specific discourse, translation quality assessment, translation.

APA Citation: Liwei Zou (2025). Evaluating the Effectiveness of Machine Translation Tools in Translating China-specific Discourse. Scientific Journal of Humanities and Social Sciences, 7(9), 52-59. https://doi.org/10.54691/30qy6287

References

Zhang, X. J., & Shao, L. The basic principles for building trustworthy machine translation systems: An engineering ethics perspective [J]. Foreign Languages and Literature, 2021, (1), p.1–8.
Salvagno, M., Taccone, F. S., & Gerli, A. G. Can artificial intelligence help for scientific writing? [J]. Critical Care, 2023, 27(1), p.75.
Yuan, Y. L. Beyond chatbots: The success of ChatGPT and its implications for linguistics [J]. Contemporary Linguistics, 2023, (5), p.633–652.
Geng, F., & Hu, J. New directions in AI-assisted post-editing: A case study using ChatGPT [J]. China Foreign Languages, 2023, (3), p.41–47.
Yang, F. C. Reflections and implications of ChatGPT for interpreters: A case study of Vietnamese legal translation [J]. Chinese Science & Technology Translators Journal, 2023, (3), p.27–30, 4.
Wen, X., & Tian, Y. L. A study on the effectiveness of ChatGPT in translating Chinese characteristic discourse [J]. Shanghai Journal of Translators, 2024, (2), p.27–34.
Wang, L. F., & Li, Z. ChatGPT accelerates the transformation of translation and foreign language education [EB/OL]. https://tech.china.com/article/20230222/022023_1226999.html. (2023-02-22) .
Zhu, G. H., & Wang, X. W. The operational model, key technologies, and future prospects of ChatGPT [J]. Journal of Xinjiang Normal University (Philosophy and Social Sciences Edition), 2023, (4), p.113–122.
Zhu, Y. H. A China-specific analysis of external publicity translation and its translation strategies [M]. Suzhou: Suzhou University Press, 2017, p.1.
Wang, J. S., Zhuang, C. Q., & Wei, Y. P. Machine translation quality assessment: Methods, applications, and outlook [J]. Foreign Languages and Literature, 2024, (5), p.135–144.