Bio
A key challenge of AI research today is for AI systems reflect the knowledge, judgment, and professional standards of domain experts. The central focus of my PhD is to advance AI systems in order for them to act as a supportive collaborator with the help of human experts insights. This alignment improves trust, reliability, and real-world usefulness, especially in high-stakes areas like software engineering. Besides human-AI scope, being a linguist, I am also interested in more traditional NLP topics such as how to tackle multilinguality or semantic ambiguity. Outside of my research, I enjoy exploring and practicing various forms of art, hiking and visiting new places.
Research Interests
- Expert-AI collaboration
- Subjective Preferences alignment
- Human evaluation of AI
Publications
2026
-
Subjective Code Preferences in Experts and Large Language Models
Anna Mokhova, Subhabrata Dutta, Iryna Gurevych, and 1 more author
arXiv preprint arXiv:2605.25296, 2026
Large Language Models (LLMs) have become increasingly popular for coding tasks, with subjective coding preferences being an essential element to adapt to programmers’ personal needs. Existing work overlooks such characteristics and mainly focuses on code correctness. In this study, we propose a typification of four subjective coding preference axes - complexity, commenting, modularity, and readability - motivated by common engineering habits and validated by 25 software engineers. We collect a dataset of 3,000 paired Python code snippets reflecting these axes, annotated by 73 experts who rate their preferences on a Likert scale. Using our dataset, we study how LLMs handle subjective coding preferences. We present 13 LLMs with pairs of solutions to the same programming task, first as textual descriptions and then as concrete code snippets. We find that models often prefer one option in natural language but the opposite when evaluating code. More consistent models (i.e., those that are coherent in their choices between deeds and words) frequently reveal positional bias: swapping the order of options changes the preferred alternative. We then use the five most consistent models to re-annotate the dataset. Compared to humans, models show polarized Likert distributions and notable divergence in ratings. A case study on GPT-5 reveals reliance on external assumptions and brittle reasoning.
