LLM2Vec abstract

piano afro-rock

April 11th, 2024suno

Lyrics

[Bridge] [Verse 1] Large decoder-only language models LLMs are the state-of-the-art models on most of today’s NLP tasks [Verse 2] Yet, the community is only slowly adopting these models for text embedding tasks which require rich contextualized representations. [Verse 3] In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. [Verse 4] We demonstrate the effectiveness of LLM2Vec by applying it to three popular LLMs [Verse 5] We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark MTEB. [Verse 6] Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. Three. unsupervised contrastive learning. []Verse 7] Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed [Verse 8] into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning.

Recommended

L'ennui sans fin
L'ennui sans fin

pop boy band entraînant harmonieux

Monday's Escape
Monday's Escape

male vocalist,rock,punk rock,skate punk,pop punk,energetic,punk,melodic hardcore,playful

Getting It
Getting It

native american flute, strings, hardcore modern edm, deep bass drops

Góngora
Góngora

dramatic, electronic, rock, romantic, guitar, jazz, sad, male voice

Trapped in the Beat Machine
Trapped in the Beat Machine

electronic syncopated new wave

Why Not
Why Not

Math rock, nuevo flamenco

Thin White Crescent Moon
Thin White Crescent Moon

avant garde acoustic dreamy

Rise Again
Rise Again

стиль классического рока. Такой стиль характеризуется мощными гитарными риффами, энергичными барабанными партиями и выра

Satu Tekad
Satu Tekad

rock and roll, arena rock, electric guitar intro, progressive, clean, rhythm, riff, hard rock, female vocal

ABE JAVASON - DIRIMU
ABE JAVASON - DIRIMU

anime, epic, experimental, fantasy, dreamy, male voice

Šokių Naktis
Šokių Naktis

euphoric, male and female vocals alternating, electronic; layered synths, pop, eurodance, pulsating bassline, high-energy

Dancing in the Rain
Dancing in the Rain

feel-good pop laid-back

scream of the lambs
scream of the lambs

danger music, fluxus, avant-garde, piano, drum

Thriller Night
Thriller Night

cinematic synth horror

Illuminatum
Illuminatum

ambient new age worldbeat

S C - 04
S C - 04

JAZZ,FEMALE SINGER, french chanson light opera,SOFT ,JOYFUL,CHILL,