From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation
#Large Language Models#Parallel Code Generation#Task-based Programming#OpenMP#C++ Parallelism#HPX#AI-assisted Development
📌 Key Takeaways
Researchers evaluated LLM capabilities for generating task-based parallel code
The study tested three types of input prompts: natural language, sequential implementations, and parallel pseudo code
Three programming frameworks were examined: OpenMP Tasking, C++ standard parallelism, and HPX
LLMs showed both strengths and weaknesses depending on problem complexity and framework
📖 Full Retelling
Researchers Linus Bantel, Moritz Strack, Alexander Strack, and Dirk Pflüger published a paper on arXiv on February 24, 2026, evaluating how Large Language Models generate efficient parallel code from different input prompts, addressing a gap in research on LLM capabilities for creating parallel programs. The paper, titled 'From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation,' explores how these models perform when generating task-based parallel code using three different types of input prompts: natural language problem descriptions, sequential reference implementations, and parallel pseudo code. The researchers focused on three programming frameworks: OpenMP Tasking, C++ standard parallelism, and the asynchronous many-task runtime HPX, each offering different levels of abstraction and control for task execution. The study evaluated LLM-generated solutions for both correctness and scalability, revealing both strengths and weaknesses of these models depending on problem complexity and the specific programming framework used. The findings have important implications for future LLM-assisted development in high-performance and scientific computing, suggesting that while LLMs show promise in code generation, their ability to create efficient parallel programs requires further development and understanding.
OpenMP is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on many platforms, instruction-set architectures and operating systems, including Solaris, AIX, FreeBSD, HP-UX, Linux, macOS, Windows and OpenHarmony. I...
A large language model (LLM) is a language model trained with self-supervised machine learning on a vast amount of text, designed for natural language processing tasks, especially language generation. The largest and most capable LLMs are generative pre-trained transformers (GPTs) that provide the c...
HPX, short for High Performance ParalleX, is a runtime system for high-performance computing. It is currently under active development by the STE||AR group at Louisiana State University. Focused on scientific computing, it provides an alternative execution model to conventional approaches such as MP...
No entity connections available yet for this article.
Original Source
--> Computer Science > Programming Languages arXiv:2602.22240 [Submitted on 24 Feb 2026] Title: From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation Authors: Linus Bantel , Moritz Strack , Alexander Strack , Dirk Pflüger View a PDF of the paper titled From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation, by Linus Bantel and Moritz Strack and Alexander Strack and Dirk Pfl\"uger View PDF HTML Abstract: Large Language Models show strong abilities in code generation, but their skill in creating efficient parallel programs is less studied. This paper explores how LLMs generate task-based parallel code from three kinds of input prompts: natural language problem descriptions, sequential reference implementations, and parallel pseudo code. We focus on three programming frameworks: OpenMP Tasking, C++ standard parallelism, and the asynchronous many-task runtime HPX. Each framework offers different levels of abstraction and control for task execution. We evaluate LLM-generated solutions for correctness and scalability. Our results reveal both strengths and weaknesses of LLMs with regard to problem complexity and framework. Finally, we discuss what these findings mean for future LLM-assisted development in high-performance and scientific computing. Comments: 12 pages, 4 figures, 2 tables, Workshop on Asynchronous Many-Task Systems and Applications 2026 Subjects: Programming Languages (cs.PL) ; Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) Cite as: arXiv:2602.22240 [cs.PL] (or arXiv:2602.22240v1 [cs.PL] for this version) https://doi.org/10.48550/arXiv.2602.22240 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Linus Bantel [ view email ] [v1] Tue, 24 Feb 2026 09:49:10 UTC (358 KB) Full-text links: Access Paper: View a PDF of the paper titled From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation, by Linus Bantel and Mori...
