KInIT at EMNLP 2024 Conference

From the 12th of November until the 16th of November, the prestigious conference on Empirical Methods in Natural Language Processing (EMNLP) took place in Miami. The conference brings together AI experts and researchers from around the world, focusing on natural language processing and fostering an environment of collaboration and networking. Two of our colleagues, Branislav Pecher and Dominik Macko, attended the conference and presented their work accepted there.

Dominik and Branislav presented quite a substantive part of the research conducted here at Kempelen Institute. The conference accepted three papers from our researchers, all of which were outcomes of our ongoing European projects – namely the paper on authorship obfuscation in multilingual machine-generated text detection (AI-CODE project), on sensitivity of learning with limited labelled data to the effects of randomness (vera.ai project) and how such undesired randomness can be addressed by our proposed mitigation technique (VIGILANT project).

We are also excited to share the atmosphere of the conference through Branislav’s eyes:

This year, we had the opportunity to attend the EMNLP’24 conference in Miami and present our work. Of all the AI fields, natural language processing has recently experienced the most significant growth, which was also evident during the conference. It was attended by a staggering 4100+ participants, from which more than 3400 attended in person. As such, it was a massive event with many networking opportunities.

The conference started with a social gathering, full of people chatting about their research and enjoying the nice Miami weather. We were also invited to a social dinner, organised in the Frost Science Museum. Besides further discussions, we entertained ourselves with visits to the aquarium and the planetarium, a screening of a short movie about black holes, and some dancing…and lots of local food.

Main Event – Keynotes

As with all similar events, the event included one keynote speaker every day of the conference. The first day was kicked off by Percy Liang, an associate professor at Stanford, who talked about open-source and its place in the era of foundation models. Professor Liang’s presentation stressed how important it is to have open-source models in the era of massive language models. Even though all the keynotes were amazing, I found this one most interesting. It can be thought of a bit as a wake-up call for all the researchers – to fight more for keeping the models open-source, as it allows for fundamental innovations and drives research forward.

The second keynote speaker was Anca Dragan, an associate professor from UC Berkeley, who works on AI Safety and Alignment at Google DeepMind. The topic of her talk was the alignment of the large language models, or in other words, how to make the LLMs conform to our human wants and be safe. The main problem is to determine how to teach the model what is considered to be a “bad thing” – even for the things the model did not encounter during training.

The final keynote was delivered by Tim Griffiths, a professor at Princeton University. The topic was the Bayes in the age of intelligent machines – or, more specifically, utilising knowledge from psychology in training LLMs. The main focus of the talk was on using Bayesian analysis to explore the failings of language models and how the priors affect the success of the models. In essence, the presentation has shown us how brittle and actually “not-so-intelligent” the current models are.

Poster Sessions – the best opportunity for networking

For us, the highlight of the conference was the poster session with many research posters and a lot of talking. It was an amazing opportunity to meet new people working on similar problems as we do, discuss our findings with them, come up with new ideas, network and potentially set up new collaborations. We participated in these poster sessions actively – Dominik presented one and Branislav two papers. In both cases, there were many people who came by to listen to our research and discuss it further. In some cases, we even gained potential new collaborators as there were researchers highly interested in our research who would want to use it further.

Volunteering – how to meet new people easily

I have also participated in the conference as a student volunteer. As always, it was a great experience – the volunteering jobs are not as hard as one would imagine. My responsibilities were mainly just answering questions from participants and dealing with problems if they came up. However, it brings many benefits – for example, as you are wearing a bright and visible T-shirt, many of the participants feel more comfortable starting a conversation with you, even about your research. I had multiple researchers stop me this way and start chatting with me.

Workshop and Tutorial – a hidden gem

Besides the main events, we have also attended the workshops and tutorials available. Although they may not be considered as prestigious as the main conference papers, the research presented during the workshop was as interesting (if not more) than the ones from the main talks. One of the strengths of the workshops is that they are focused, so meeting researchers dealing with the same specific research areas and problems as yourself is very easy. Therefore, almost half of the most interesting research discussions and ideas came from discussing the workshop papers during the poster sessions.