jún 6. 2024

KInIT Natural Language Processing Summer School

The first KInIT Summer school on NLP will take place on September 2nd, 2024 as a 1-day long event, focusing primarily on university students to provide an introduction to natural language processing (NLP) – a subfield of AI with tremendous impact and potential to improve our lives. It will take place at the Kempelen Institute for Intelligent Technologies (KInIT) in Bratislava under the supervision of experts from KInIT and partner institutions. 

The Summer school will welcome university students (any year of study), smart high school students will be considered as well. The event is free of charge. There will be a selection process based on the motivation letter as the capacity of the venue is limited.

The focus of the summer school will be on introducing the field of natural language processing and tasks it incorporates. Next, we will delve into the neural networks and fundamental approaches that most of NLP applications today are built on. Finally, a hands-on session aimed at language model fine-tuning will enable students to engage with the technology, apply theoretical knowledge and develop practical skills.

Selected students will be invited to participate in the selected events from DisAI Summer School on trustworthy, multilingual and multimodal AI, offering advanced topics of AI and NLP and involving internationally recognized speakers from Google, Meta, DFKI, CERTH and University of Copenhagen. The 4-day long DisAI Summer school follows the Summer school on NLP.

Kempelen institute of intelligent technologies (KInIT) is an independent, non-profit institute dedicated to intelligent technology research. On our mission of interconnecting academia and industry, we bring together and nurture experts in artificial intelligence and other areas of computer science, with connections to other disciplines.

When?

  • 2nd September 2024, 8:30-16:00 CET

Programme: 

  • 8:30 Breakfast
  • 9:00 Intro to NLP, NLP tasks (Marián Šimko, KInIT)
    In this talk we will introduce Natural Language Processing as an exciting field at the crossroads of artificial intelligence, linguistics and computer science. We will briefly outline the main tasks and provide historical perspective on recent developments. We will discuss recent breakthrough applications and outline limitations and ethical perspectives of current approaches.
  • 10:00 Intro to Deep Learning (Michal Gregor, KInIT)
    Brief introduction to deep neural networks and deep learning for the uninitiated. We will make a very brief dive into the basic principles of how artificial neural networks learn, what components are needed to make it work in networks with many layers and what advantages that brings us in terms of global generalization. Finally, we will look into why well-designed inductive preferences are key to good results.
  • 11:30 Lunch
  • 12:30 Deep Learning for Sequential Data: LSTMs, Transformers, et al. (Marek Šuppa, FMFI UK, Slido)
    This talk will discuss the application of deep learning to sequential data, primarily to natural language. It will cover architectures designed for this kind of data, especially the transformer architecture (which is now being applied not only to natural language, but across different modalities including audio, vision, etc.). Finally, it will address several language-specific concepts such as tokenization, token/word embeddings, etc.
  • 13:30 Coffee Break
  • 14:00 Fine-tuning language models – hands on (Martin Tamajka, KInIT)
    Description: In this workshop, we’ll look at tools commonly used to effectively fine-tune language models so that they better fit your data and task. In the hands-on part, you’ll fine-tune one or several language models and evaluate how the fine-tuning helped to increase their performance.
  • 16:00 Goodbye
Register

Partners

How to attend?

The event will take place on 2nd September 2024 in the KInIT offices at The Spot on the 6th floor.

The Spot, Sky Park Offices, Bottova 7939/2A, 811 09 Bratislava

About us:

Natural language processing (NLP) is the intersection of information technology and linguistics. It is concerned with processing the huge amounts of unstructured natural language data that emerge at lightning speed in the digital era – the age of social networks. We research and deliver novel methods to improve NLP tasks of different types, while covering multiple stages of text processing.

Our research combines various approaches, from linguistic and statistical to machine learning and deep learning. We employ novel language models that take advantage of recent advances in the field, while covering a variety of application domains. We focus on open problems related to text classification, information extraction, sentiment analysis and text generation. We look for applications of transfer learning and improving the interpretability of NLP models. Our work includes the processing of low-resource languages, such as Slovak.

Read more information about our research here.