LLM2Vec abstract

piano afro-rock

April 11th, 2024suno

Lyrics

[Bridge] [Verse 1] Large decoder-only language models LLMs are the state-of-the-art models on most of today’s NLP tasks [Verse 2] Yet, the community is only slowly adopting these models for text embedding tasks which require rich contextualized representations. [Verse 3] In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. [Verse 4] We demonstrate the effectiveness of LLM2Vec by applying it to three popular LLMs [Verse 5] We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark MTEB. [Verse 6] Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. Three. unsupervised contrastive learning. []Verse 7] Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed [Verse 8] into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning.

Recommended

Nummulites az óriás egysejtű
Nummulites az óriás egysejtű

poetic floating piano-driven

Kaffee oder Möööt? Am Morgen
Kaffee oder Möööt? Am Morgen

German, bayrisch, musikantenstadel

Sending Signs
Sending Signs

energetic, rock, beat, pop, house, upbeat

Верните в моду любовь
Верните в моду любовь

male vocals, pop, electro, techno, house, trance, indie pop

Moonlit Night
Moonlit Night

progesive rock metal slow bass

Lost in Screens
Lost in Screens

female vocalist,electronic,dance,dance-pop,electropop,pop rock,electroclash,rock,anthemic

Rainfall in the Shadows
Rainfall in the Shadows

haunting atmospheric viking folk

The Sea Calls
The Sea Calls

Acoustic guitar, mandolin, subtle violin, gentle banjo, rythmic

Ocean Deep Love
Ocean Deep Love

Metalcore post hardcore melodic,catchy,emotional, male voice, emo

Chair song
Chair song

simple, playful, for little children, female vocal

Miles and Miles3
Miles and Miles3

Emotional soul and gospel, building up, male vocals, choirs

Angel to Your Demons
Angel to Your Demons

melodic dubstep

Dubai Skies Unfold
Dubai Skies Unfold

male vocalist,arabic music,melodic

Aigle, voyageur de l'espace.
Aigle, voyageur de l'espace.

electro, electronic, pop, rock, metal

idk 1
idk 1

sad violin, cello, jack, skellington, film score, theatre and entertaiment, dark

Rise From the Ashes
Rise From the Ashes

anthemic rap experimental

Just friends
Just friends

Acoustic guitar, singer songwriter, female, nostalgic, electric guitar, Americana

Music
Music

opera, drum and bass, electric guitar