Authors - Shruthi D, Chethan H. K, Agughasi Victor Ikechukwu Abstract - A set of encoder-decoder large language models (LLMs) called CodeT5+ was created for a range of code-related applications. It presents adaptable architecture that may be tailored to many downstream uses, including text-to-code retrieval, code creation, completion, and math programming. Text-code matching, span denoising, contrastive learning, and causal language modeling spanning unimodal and bimodal multilingual code corpora are among the varied collection of pretrained objectives used to train the model. Using frozen off-the-shelf LLMs for initialization is a significant breakthrough that enables the models to scale effectively without requiring training from scratch. The instruction-tuned CodeT5+ 200m model outperforms current benchmarks and records better state-of-the-art results in code summarization among the variants. The performance metrics were impressive: ROUGE-1 at 78.5%, ROUGE-2 at 66.2%, ROUGE-L at 77.4%, BLEU-4 at 56.3%, METEOR at 64.7% and CodeBLEU at 71.5%. These outcomes demonstrate how well Co-deT5+ comprehends and produces code for a variety of programming tasks.
