{"id":29141,"date":"2023-10-19T14:48:28","date_gmt":"2023-10-19T12:48:28","guid":{"rendered":"https:\/\/kinit.sk\/modern-unsupervised-learning-can-we-bootstrap-our-own-latent\/"},"modified":"2023-10-19T15:48:08","modified_gmt":"2023-10-19T13:48:08","slug":"modern-unsupervised-learning-can-we-bootstrap-our-own-latent","status":"publish","type":"post","link":"https:\/\/kinit.sk\/sk\/modern-unsupervised-learning-can-we-bootstrap-our-own-latent\/","title":{"rendered":"Modern Unsupervised Learning – Can We Bootstrap Our Own Latent?\u00a0"},"content":{"rendered":"
\n

This blog post is a part of the <\/em>EEML Summer School 2023<\/em><\/a> series. It\u2019s a series of impressions that we, as doctoral students, got from attending the EEML Summer School 2023 in Ko\u0161ice, Slovakia. <\/em><\/p>\n<\/div>\n\n

\n
\"\"
Figure 1: The motivation picture for bootstrapping<\/em> comes from the idiom \u201cto pull oneself up by one\u2019s own bootstraps\u201d. Source<\/a>. <\/figcaption><\/figure>\n<\/div>\n\n
\n

AI solutions, mainly deep neural networks nowadays, are often used to create useful representations of the problem we have at hand. These can, for example, be representations of images, videos, sounds, text, user behavior, behavior of programs, and much much more. In the past, supervised solutions achieved much better results in terms of good representations of these modalities. <\/p>\n<\/div>\n\n

\n

However, the situation is slowly changing with the advent of new unsupervised learning solutions. In this post, we will introduce one of them – Bootstrap Your Own Latent (BYOL<\/a>). BYOL is inspired by previously published contrastive learning methods, although it cannot be strictly considered a contrastive learning method. It was introduced to us at the Eastern European Machine Learning Summer School 2023 by one of its authors, Michal Valko<\/a>. <\/p>\n<\/div>\n\n

\n

Contrastive learning<\/h3>\n<\/div>\n\n
\n

Typical contrastive learning methods work based on a relatively simple concept. They repulse different images (negative pairs) while attracting the same image\u2019s two views (positive pairs) [1]. Negative pairs are primarily used so that the learned solution (data representation) does not degenerate into a collapsed solution – i.e. where the same vector would represent all images. <\/p>\n<\/div>\n\n

\n

Typically, one would require a lot of negative samples in each training step. This is because there are, let\u2019s say, much fewer ways of how one can be a dog<\/em> than the ways that one can <\/em>not <\/em><\/strong>be a dog<\/em>. There are, for example, other animals, like cats, mice, and birds (which are not dogs), but there are also many different other things that are not dogs – like cars, houses, mountains, lakes, etc. Often, to get a good representation, the method needs to understand these differences. An example of what pairs could be used in 1 round (batch) of training a contrastive learning method is depicted in Figure 2. <\/p>\n<\/div>\n\n

\n
\"\"
Figure 2:  Depiction of positive and negative pairs in a contrastive learning (CL) setting. On the left we have a self-supervised CL context where image labels are not utilized. On the right, supervised CL is depicted where label information is used. Source<\/a><\/figcaption><\/figure>\n<\/div>\n\n
\n

Bootstrap Your Own Latent (BYOL)<\/h3>\n<\/div>\n\n
\n

Bootstrap Your Own Latent (BYOL<\/a>) is different from typical contrastive learning methods because it does not utilize any<\/em> negative pairs. This could help researchers working with other data modalities than images because finding the \u201cright\u201d negative examples for speech, sound, text, or other kinds of data could often be challenging<\/strong>. The authors of BYOL worked with images. So to make things easier, when talking about data in this article, we will talk primarily about images. <\/p>\n<\/div>\n\n

\n

The main idea of BYOL is to use bootstrapping<\/em> <\/strong>(in its idiomatic sense) <\/strong>– i.e., to gradually improve upon itself (its representation of images) without some new external guidance \u2013 a self-learning, continual self-improving process<\/strong>. The hope is that such a bootstrapping approach will help avoid solution collapse. The term bootstrapping originates from the English idiom \u201cto pull oneself by their own bootstraps\u201d \u2013 meaning that a person should be able to improve by themselves without external help. This is also why pulling boots by the bootstraps is the headline image of our blog post (Figure 1). <\/p>\n<\/div>\n\n

\n

Specifically for BYOL, bootstrapping is done on the projections of image representations. This is achieved by utilizing two neural networks – one online<\/em> network and one target<\/em> network. Figure 3 shows the architecture of BYOL. The BYOL process is also explained in the figure description. The bootstrapping works in a way that the online network tries to predict the projection of the target network. Only the online network<\/em> receives gradient updates. The target network\u2019s weights are set as the exponential moving average of the sequence of weights of the online network.<\/p>\n<\/div>\n\n

\n
\"\"
Figure 3: The architecture of BYOL. One image gets transformed into two views. Afterwards, one image gets fed into the online network and one into the target network. Their representations and projections from the respective networks are extracted and then the online network tries to predict as best as possible the projection from the target network. Loss gets computed and a gradient update of the online network is performed. Source<\/a><\/figcaption><\/figure>\n<\/div>\n\n
\n

 <\/p>\n<\/div>\n\n

\n

BYOL\u2019s results<\/h3>\n<\/div>\n\n
\n

BYOL achieves results that are on par, if not better than, the state-of-the-art (SotA) solutions – like SimCLR (a strong baseline) and MoCo. Both MoCo and SimCLR are unsupervised baselines. On ImageNet with linear evaluation, BYOL achieved the best results out of the evaluated unsupervised methods for various levels of the number of parameters. The evaluation can be found in Figure 4. Its results reach very close to the Supervised solutions, which is a tough baseline to beat since these solutions have the advantage of utilizing image labels in their learning process – a luxury that BYOL does without.<\/p>\n<\/div>\n\n

\n
\"\"
Figure 4: The evaluation of BYOL and other baseline SotA methods on the ImageNet dataset with linear evaluation on the Top-1 accuracy. Source<\/a><\/figcaption><\/figure>\n<\/div>\n\n
\n

BYOL-Explore: Moving beyond image representations<\/h3>\n<\/div>\n\n
\n

Enter the world of reinforcement learning (\u201cRL\u201d in short). In such a world, we try to create an autonomous agent that will be able to make decisions on its own based on its environment and the target goal that they have. There are many applications of RL agents in specific environments –  such as AlphaZero for playing chess, AlphaGo for playing Go, or even OpenAI Five for playing Dota 2. For BYOL-Explore, we consider RL agents that act in computer games. For an agent to be effective in such a world, it needs to have some model of the world (i.e. the game) – a \u201cworld model\u201d. In these complex environments, it is completely infeasible for an agent to explore everything and go to every place. Therefore, the agent needs to learn to intelligently prioritize the areas where it expects to achieve a better reward. <\/p>\n<\/div>\n\n

\n

BYOL-Explore is one such instantiation of an RL agent. It instantiates its world model by using BYOL. Afterwards, it improves on the standard BYOL model using curiosity-driven exploration of the game. This can also be considered a form of bootstrapping <\/em>– the term we used to explain the learning process for the simple BYOL model itself. In this way, BYOL-Explore slowly trains itself to recognize what is interesting and what should be explored – a crucial thing that needs to be done to avoid path explosion (the explosion of the possible combinations of environments and actions) and move towards the goal. <\/p>\n<\/div>\n\n

\n

The improvement of the model is done in the following way: <\/p>\n<\/div>\n\n

\n