ZBL2W at SemEval-2023 Task 9: A Multilingual Fine-tuning Model with Data Augmentation for Tweet Intimacy Analysis

This paper describes our system used in the SemEval-2023 Task 9 Multilingual Tweet Intimacy Analysis. There are two key challenges in this task: the complexity of multilingual and zero-shot cross-lingual learning, and the difficulty of semantic mining of tweet intimacy. To solve the above problems, our system extracts contextual representations from the pretrained language models, XLM-T, and employs various optimization methods, including adversarial training, data augmentation, ordinal regression loss and special training strategy. Our system ranked 14th out of 54 participating teams on the leaderboard and ranked 10th on predicting languages not in the training data. Our code is available on Github.

• Publication year

  2023

• Venue

  International Workshop on Semantic Evaluation

• Publication date

  Unknown publication date

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/2023.semeval-1.106
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• SemEval-2023 Task 9: Multilingual Tweet Intimacy Analysis

  2022cited by this paper
• HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity

  2022cited by this paper
• XLM-T: A Multilingual Language Model Toolkit for Twitter

  2021cited by this paper
• Quantifying Intimacy in Language

  2020cited by this paper
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers

  2020cited by this paper
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

  2019cited by this paper
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

  2019cited by this paper
• Cross-lingual Language Model Pretraining

  2019cited by this paper
• TinyBERT: Distilling BERT for Natural Language Understanding

  2019cited by this paper
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

  2019cited by this paper
• Adversarial Training Methods for Semi-Supervised Text Classification

  2016influential reference
• Distilling the Knowledge in a Neural Network

  2015cited by this paper
• On the Consistency of Ordinal Regression Methods

  2014cited by this paper
• Explaining and Harnessing Adversarial Examples

  2014cited by this paper
• Zero-Shot Learning Through Cross-Modal Transfer

  2013cited by this paper
• Loss Functions for Preference Levels: Regression with Discrete Ordered Labels

  2005cited by this paper

• mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans

  2024cites this paper
• SemEval-2023 Task 9: Multilingual Tweet Intimacy Analysis

  2022cites this paper