| ---
|
| language: en
|
| tags:
|
| - code
|
| - semantic-search
|
| - jepa
|
| - code-search
|
| license: mit
|
| datasets:
|
| - claudios/code_search_net
|
| metrics:
|
| - mrr
|
| ---
|
|
|
| # Repo-JEPA: Semantic Code Navigator (SOTA 0.90 MRR)
|
|
|
| A **Joint Embedding Predictive Architecture** (JEPA) for semantic code search, trained on 411,000 real Python functions using an NVIDIA H100.
|
|
|
| ## 🏆 Performance
|
|
|
| Tested on 1,000 unseen real-world Python functions from CodeSearchNet.
|
|
|
| | Metric | Result | Target |
|
| |--------|--------|--------|
|
| | **MRR** | **0.9052** | 0.60 |
|
| | **Hits@1** | **86.2%** | - |
|
| | **Hits@5** | **95.9%** | - |
|
| | **Hits@10** | **97.3%** | - |
|
| | **Median Rank** | **1.0** | - |
|
|
|
| ## 🧩 Usage (AutoModel)
|
|
|
| ```python
|
| from transformers import AutoModel, AutoTokenizer
|
|
|
| # 1. Load Model
|
| model = AutoModel.from_pretrained("uddeshya-k/RepoJepa", trust_remote_code=True)
|
| tokenizer = AutoTokenizer.from_pretrained("microsoft/codebert-base")
|
|
|
|
|
| # 2. Encode Code
|
| code = "def handle_login(user): return auth.verify(user)"
|
| code_embed = model.encode_code(**tokenizer(code, return_tensors="pt"))
|
|
|
| # 3. Encode Query
|
| query = "how to authenticate users?"
|
| query_embed = model.encode_query(**tokenizer(query, return_tensors="pt"))
|
|
|
| # 4. Search
|
| similarity = (code_embed @ query_embed.T).item()
|
| print(f"Similarity: {similarity:.4f}")
|
| ```
|
|
|
| ## 🏗️ Technical Details
|
|
|
| - **Backbone**: CodeBERT (RoBERTa-style)
|
| - **Loss**: VICReg (Variance-Invariance-Covariance Regularization)
|
| - **Hardware**: NVIDIA H100 PCIe (80GB VRAM)
|
| - **Optimizer**: AdamW + OneCycleLR
|
|
|
