{"id":41342,"date":"2026-04-02T11:01:09","date_gmt":"2026-04-02T09:01:09","guid":{"rendered":"https:\/\/kinit.sk\/?post_type=publication&#038;p=41342"},"modified":"2026-05-14T14:48:33","modified_gmt":"2026-05-14T12:48:33","slug":"peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark","status":"publish","type":"publication","link":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/","title":{"rendered":"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark"},"content":{"rendered":"<div id=\"\" class=\"element core-paragraph\">\n<p>Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by reducing the number of trainable parameters while maintaining strong downstream performance. Despite the advances in PEFT methods, current evaluations remain limited (in terms of evaluated models and datasets) and difficult to reproduce. To bridge this gap, we introduce PEFT-Bench, a unified end-to-end benchmark for evaluating diverse PEFT methods on autoregressive LLMs. We demonstrate its usage across 27 NLP datasets and 7 PEFT methods. To account for different PEFT training and inference factors, we also introduce the PEFT Soft Cost Penalties (PSCP) metric, which takes trainable parameters, inference speed, and training memory usage into account.<br><br><\/p>\n<\/div>\n\n<div id=\"\" class=\"element core-paragraph\">\n<p><strong>Cite<\/strong>: <em>Robert Belanec, Branislav Pecher, Ivan Srba, and Maria Bielikova. 2026.&nbsp;<a href=\"https:\/\/aclanthology.org\/2026.eacl-long.140\/\">PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark<\/a>. In&nbsp;<em>Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)<\/em>, pages 3035\u20133054, Rabat, Morocco. Association for Computational Linguistics.<\/em><\/p>\n<\/div>","protected":false},"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"categories":[78,236,542],"class_list":["post-41342","publication","type-publication","status-publish","hentry","category-web-user-data-processing-sk","category-bielikovam-sk","category-2026-sk"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/\" \/>\n<meta property=\"og:locale\" content=\"sk_SK\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT\" \/>\n<meta property=\"og:description\" content=\"Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT)...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/\" \/>\n<meta property=\"og:site_name\" content=\"KInIT\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-14T12:48:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/kinit.sk\/wp-content\/uploads\/2021\/03\/KINIT_Sharepic.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@kinit\" \/>\n<meta name=\"twitter:label1\" content=\"Predpokladan\u00fd \u010das \u010d\u00edtania\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 min\u00fata\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/publikacia\\\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\\\/\",\"url\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/publikacia\\\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\\\/\",\"name\":\"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/kinit.sk\\\/#website\"},\"datePublished\":\"2026-04-02T09:01:09+00:00\",\"dateModified\":\"2026-05-14T12:48:33+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/publikacia\\\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\\\/#breadcrumb\"},\"inLanguage\":\"sk-SK\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/kinit.sk\\\/sk\\\/publikacia\\\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/publikacia\\\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Web &amp; User Data Processing\",\"item\":\"https:\\\/\\\/kinit.sk\\\/sk\\\/category\\\/web-user-data-processing-sk\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/kinit.sk\\\/#website\",\"url\":\"https:\\\/\\\/kinit.sk\\\/\",\"name\":\"KInIT\",\"description\":\"Vyu\u017e\u00edvame v\u00fdskum pre \u013eud\u00ed a priemysel\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/kinit.sk\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"sk-SK\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/","og_locale":"sk_SK","og_type":"article","og_title":"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT","og_description":"Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT)...","og_url":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/","og_site_name":"KInIT","article_modified_time":"2026-05-14T12:48:33+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/kinit.sk\/wp-content\/uploads\/2021\/03\/KINIT_Sharepic.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@kinit","twitter_misc":{"Predpokladan\u00fd \u010das \u010d\u00edtania":"1 min\u00fata"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/","url":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/","name":"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark - KInIT","isPartOf":{"@id":"https:\/\/kinit.sk\/#website"},"datePublished":"2026-04-02T09:01:09+00:00","dateModified":"2026-05-14T12:48:33+00:00","breadcrumb":{"@id":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/#breadcrumb"},"inLanguage":"sk-SK","potentialAction":[{"@type":"ReadAction","target":["https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/kinit.sk\/sk\/publikacia\/peft-bench-a-parameter-efficient-fine-tuning-methods-benchmark\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/kinit.sk\/sk\/"},{"@type":"ListItem","position":2,"name":"Web &amp; User Data Processing","item":"https:\/\/kinit.sk\/sk\/category\/web-user-data-processing-sk\/"},{"@type":"ListItem","position":3,"name":"PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark"}]},{"@type":"WebSite","@id":"https:\/\/kinit.sk\/#website","url":"https:\/\/kinit.sk\/","name":"KInIT","description":"Vyu\u017e\u00edvame v\u00fdskum pre \u013eud\u00ed a priemysel","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/kinit.sk\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"sk-SK"}]}},"_links":{"self":[{"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/publication\/41342","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/types\/publication"}],"version-history":[{"count":1,"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/publication\/41342\/revisions"}],"predecessor-version":[{"id":42450,"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/publication\/41342\/revisions\/42450"}],"wp:attachment":[{"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/media?parent=41342"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kinit.sk\/sk\/wp-json\/wp\/v2\/categories?post=41342"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}