Image-based Joint-Embedding Predictive Architecture (I-JEPA) offers a promising approach to visual self-supervised learning through masked feature prediction. However with the inherent visual uncertainty at masked positions, feature prediction remains challenging and may fail to learn semantic representations. In this work, we propose Text-Conditional JEPA (TC-JEPA) that uses image captions to reduce the prediction uncertainty. Specifically, we modulate the predicted patch features using a fine-grained text conditioner that computes sparse cross-attention over input text tokens. With such… Read More
Text-Conditional JEPA for Learning Semantically Rich Visual Representations
Image-based Joint-Embedding Predictive Architecture (I-JEPA) offers a promising approach to visual self-supervised learning through masked feature prediction. However with the inherent visual uncertainty at masked positions, feature prediction remains challenging and may fail to learn semantic representations. In this work, we propose Text-Conditional JEPA (TC-JEPA) that uses image captions to reduce the prediction uncertainty. Specifically, we modulate the predicted patch features using a fine-grained text conditioner that computes sparse cross-attention over input text tokens. With such…
Related Posts
Setting Up a Google Colab AI-Assisted Coding Environment That Actually Works
This article focuses on Google Colab , an increasingly popular, free, and accessible, cloud-based Python…
AI Is Reshaping Developer Career Paths
This article is part of a series on the Sens-AI Framework—practical habits for learning and…
If You’ve Never Broken It, You Don’t Really Know It
The following article originally appeared on Medium and is being republished here with the author’s permission. There’s…