Hugging Face's logo

tomaarsen
/
splade-cocondenser-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
bert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:3011496
loss:SpladeLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
tomaarsen HF Staff commited on
Commit
8b837da
·
verified ·
1 Parent(s): df5beb0

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1343 -0
  3. config.json +26 -0
  4. config_sentence_transformers.json +11 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +58 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1343 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:3011496
12
+ - loss:SpladeLoss
13
+ base_model: Luyu/co-condenser-marco
14
+ widget:
15
+ - source_sentence: how much percent of alcohol is in scotch?
16
+ sentences:
17
+ - Our 24-hour day comes from the ancient Egyptians who divided day-time into 10
18
+ hours they measured with devices such as shadow clocks, and added a twilight hour
19
+ at the beginning and another one at the end of the day-time, says Lomb. "Night-time
20
+ was divided in 12 hours, based on the observations of stars.
21
+ - After distillation, a Scotch Whisky can be anywhere between 60-75% ABV, with American
22
+ Whiskey rocketing right into the 90% region. Before being placed in casks, Scotch
23
+ is usually diluted to around 63.5% ABV (68% for grain); welcome to the stage cask
24
+ strength Whisky.
25
+ - Money For Nothing. In season four Dominic West, the ostensible star of the series,
26
+ requested a reduced role so that he could spend more time with his family in London.
27
+ On the show it was explained that Jimmy McNulty had taken a patrol job which required
28
+ less strenuous work.
29
+ - source_sentence: what are the major causes of poor listening?
30
+ sentences:
31
+ - The four main causes of poor listening are due to not concentrating, listening
32
+ too hard, jumping to conclusions and focusing on delivery and personal appearance.
33
+ Sometimes we just don't feel attentive enough and hence don't concentrate.
34
+ - That's called being idle. “System Idle Process” is the software that runs when
35
+ the computer has absolutely nothing better to do. It has the lowest possible priority
36
+ and uses as few resources as possible, so that if anything at all comes along
37
+ for the CPU to work on, it can.
38
+ - 'No alcohol wine: how it''s made It''s not easy. There are three main methods
39
+ currently in use. Vacuum distillation sees alcohol and other volatiles removed
40
+ at a relatively low temperature (25°C-30°C), with aromatics blended back in afterwards.'
41
+ - source_sentence: are jess and justin still together?
42
+ sentences:
43
+ - Download photos and videos to your device On your iPhone, iPad, or iPod touch,
44
+ tap Settings > [your name] > iCloud > Photos. Then select Download and Keep Originals
45
+ and import the photos to your computer. On your Mac, open the Photos app. Select
46
+ the photos and videos you want to copy.
47
+ - Later, Justin reunites with Jessica at prom and the two get back together. ...
48
+ After a tearful goodbye to Jessica, the Jensens, and his friends, Justin dies
49
+ just before graduation.
50
+ - Incumbent president Muhammadu Buhari won his reelection bid, defeating his closest
51
+ rival Atiku Abubakar by over 3 million votes. He was issued a Certificate of Return,
52
+ and was sworn in on May 29, 2019, the former date of Democracy Day (Nigeria).
53
+ - source_sentence: when humans are depicted in hindu art?
54
+ sentences:
55
+ - 'Answer: Humans are depicted in Hindu art often in sensuous and erotic postures.'
56
+ - Bettas are carnivores. They require foods high in animal protein. Their preferred
57
+ diet in nature includes insects and insect larvae. In captivity, they thrive on
58
+ a varied diet of pellets or flakes made from fish meal, as well as frozen or freeze-dried
59
+ bloodworms.
60
+ - An active continental margin is found on the leading edge of the continent where
61
+ it is crashing into an oceanic plate. ... Passive continental margins are found
62
+ along the remaining coastlines.
63
+ - source_sentence: what is the difference between 18 and 20 inch tires?
64
+ sentences:
65
+ - '[''Alienware m17 R3. The best gaming laptop overall offers big power in slim,
66
+ redesigned chassis. ... '', ''Dell G3 15. ... '', ''Asus ROG Zephyrus G14. ...
67
+ '', ''Lenovo Legion Y545. ... '', ''Alienware Area 51m. ... '', ''Asus ROG Mothership.
68
+ ... '', ''Asus ROG Strix Scar III. ... '', ''HP Omen 17 (2019)'']'
69
+ - So extracurricular activities are just activities that you do outside of class.
70
+ The Common App says that extracurricular activities "include arts, athletics,
71
+ clubs, employment, personal commitments, and other pursuits."
72
+ - The only real difference is a 20" rim would be more likely to be damaged, as you
73
+ pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the
74
+ availability of tires will likely be much more limited for the larger rim. ...
75
+ Tire selection is better for 18" wheels than 20" wheels.
76
+ datasets:
77
+ - sentence-transformers/gooaq
78
+ pipeline_tag: feature-extraction
79
+ library_name: sentence-transformers
80
+ metrics:
81
+ - dot_accuracy@1
82
+ - dot_accuracy@3
83
+ - dot_accuracy@5
84
+ - dot_accuracy@10
85
+ - dot_precision@1
86
+ - dot_precision@3
87
+ - dot_precision@5
88
+ - dot_precision@10
89
+ - dot_recall@1
90
+ - dot_recall@3
91
+ - dot_recall@5
92
+ - dot_recall@10
93
+ - dot_ndcg@10
94
+ - dot_mrr@10
95
+ - dot_map@100
96
+ co2_eq_emissions:
97
+ emissions: 1032.3672234821006
98
+ energy_consumed: 2.655934941117104
99
+ source: codecarbon
100
+ training_type: fine-tuning
101
+ on_cloud: false
102
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
103
+ ram_total_size: 31.777088165283203
104
+ hours_used: 9.368
105
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
106
+ model-index:
107
+ - name: splade-cocondenser trained on GooAQ
108
+ results:
109
+ - task:
110
+ type: sparse-information-retrieval
111
+ name: Sparse Information Retrieval
112
+ dataset:
113
+ name: NanoClimateFEVER
114
+ type: NanoClimateFEVER
115
+ metrics:
116
+ - type: dot_accuracy@1
117
+ value: 0.18
118
+ name: Dot Accuracy@1
119
+ - type: dot_accuracy@3
120
+ value: 0.38
121
+ name: Dot Accuracy@3
122
+ - type: dot_accuracy@5
123
+ value: 0.52
124
+ name: Dot Accuracy@5
125
+ - type: dot_accuracy@10
126
+ value: 0.62
127
+ name: Dot Accuracy@10
128
+ - type: dot_precision@1
129
+ value: 0.18
130
+ name: Dot Precision@1
131
+ - type: dot_precision@3
132
+ value: 0.14
133
+ name: Dot Precision@3
134
+ - type: dot_precision@5
135
+ value: 0.12
136
+ name: Dot Precision@5
137
+ - type: dot_precision@10
138
+ value: 0.08
139
+ name: Dot Precision@10
140
+ - type: dot_recall@1
141
+ value: 0.115
142
+ name: Dot Recall@1
143
+ - type: dot_recall@3
144
+ value: 0.19833333333333333
145
+ name: Dot Recall@3
146
+ - type: dot_recall@5
147
+ value: 0.2683333333333333
148
+ name: Dot Recall@5
149
+ - type: dot_recall@10
150
+ value: 0.33233333333333326
151
+ name: Dot Recall@10
152
+ - type: dot_ndcg@10
153
+ value: 0.25936082036566754
154
+ name: Dot Ndcg@10
155
+ - type: dot_mrr@10
156
+ value: 0.3062460317460317
157
+ name: Dot Mrr@10
158
+ - type: dot_map@100
159
+ value: 0.20719224548153503
160
+ name: Dot Map@100
161
+ - task:
162
+ type: sparse-information-retrieval
163
+ name: Sparse Information Retrieval
164
+ dataset:
165
+ name: NanoDBPedia
166
+ type: NanoDBPedia
167
+ metrics:
168
+ - type: dot_accuracy@1
169
+ value: 0.6
170
+ name: Dot Accuracy@1
171
+ - type: dot_accuracy@3
172
+ value: 0.78
173
+ name: Dot Accuracy@3
174
+ - type: dot_accuracy@5
175
+ value: 0.84
176
+ name: Dot Accuracy@5
177
+ - type: dot_accuracy@10
178
+ value: 0.92
179
+ name: Dot Accuracy@10
180
+ - type: dot_precision@1
181
+ value: 0.6
182
+ name: Dot Precision@1
183
+ - type: dot_precision@3
184
+ value: 0.52
185
+ name: Dot Precision@3
186
+ - type: dot_precision@5
187
+ value: 0.5120000000000001
188
+ name: Dot Precision@5
189
+ - type: dot_precision@10
190
+ value: 0.452
191
+ name: Dot Precision@10
192
+ - type: dot_recall@1
193
+ value: 0.05522786915214328
194
+ name: Dot Recall@1
195
+ - type: dot_recall@3
196
+ value: 0.11018533697480869
197
+ name: Dot Recall@3
198
+ - type: dot_recall@5
199
+ value: 0.1586380992797861
200
+ name: Dot Recall@5
201
+ - type: dot_recall@10
202
+ value: 0.28717168510493385
203
+ name: Dot Recall@10
204
+ - type: dot_ndcg@10
205
+ value: 0.5254041819320687
206
+ name: Dot Ndcg@10
207
+ - type: dot_mrr@10
208
+ value: 0.6988888888888889
209
+ name: Dot Mrr@10
210
+ - type: dot_map@100
211
+ value: 0.3939831545534725
212
+ name: Dot Map@100
213
+ - task:
214
+ type: sparse-information-retrieval
215
+ name: Sparse Information Retrieval
216
+ dataset:
217
+ name: NanoFEVER
218
+ type: NanoFEVER
219
+ metrics:
220
+ - type: dot_accuracy@1
221
+ value: 0.74
222
+ name: Dot Accuracy@1
223
+ - type: dot_accuracy@3
224
+ value: 0.88
225
+ name: Dot Accuracy@3
226
+ - type: dot_accuracy@5
227
+ value: 0.94
228
+ name: Dot Accuracy@5
229
+ - type: dot_accuracy@10
230
+ value: 0.94
231
+ name: Dot Accuracy@10
232
+ - type: dot_precision@1
233
+ value: 0.74
234
+ name: Dot Precision@1
235
+ - type: dot_precision@3
236
+ value: 0.29333333333333333
237
+ name: Dot Precision@3
238
+ - type: dot_precision@5
239
+ value: 0.19999999999999996
240
+ name: Dot Precision@5
241
+ - type: dot_precision@10
242
+ value: 0.102
243
+ name: Dot Precision@10
244
+ - type: dot_recall@1
245
+ value: 0.7166666666666667
246
+ name: Dot Recall@1
247
+ - type: dot_recall@3
248
+ value: 0.8266666666666667
249
+ name: Dot Recall@3
250
+ - type: dot_recall@5
251
+ value: 0.9166666666666667
252
+ name: Dot Recall@5
253
+ - type: dot_recall@10
254
+ value: 0.9233333333333333
255
+ name: Dot Recall@10
256
+ - type: dot_ndcg@10
257
+ value: 0.8283955451135206
258
+ name: Dot Ndcg@10
259
+ - type: dot_mrr@10
260
+ value: 0.8096666666666669
261
+ name: Dot Mrr@10
262
+ - type: dot_map@100
263
+ value: 0.7933820346320346
264
+ name: Dot Map@100
265
+ - task:
266
+ type: sparse-information-retrieval
267
+ name: Sparse Information Retrieval
268
+ dataset:
269
+ name: NanoFiQA2018
270
+ type: NanoFiQA2018
271
+ metrics:
272
+ - type: dot_accuracy@1
273
+ value: 0.34
274
+ name: Dot Accuracy@1
275
+ - type: dot_accuracy@3
276
+ value: 0.46
277
+ name: Dot Accuracy@3
278
+ - type: dot_accuracy@5
279
+ value: 0.52
280
+ name: Dot Accuracy@5
281
+ - type: dot_accuracy@10
282
+ value: 0.62
283
+ name: Dot Accuracy@10
284
+ - type: dot_precision@1
285
+ value: 0.34
286
+ name: Dot Precision@1
287
+ - type: dot_precision@3
288
+ value: 0.20666666666666667
289
+ name: Dot Precision@3
290
+ - type: dot_precision@5
291
+ value: 0.15200000000000002
292
+ name: Dot Precision@5
293
+ - type: dot_precision@10
294
+ value: 0.09599999999999997
295
+ name: Dot Precision@10
296
+ - type: dot_recall@1
297
+ value: 0.1855793650793651
298
+ name: Dot Recall@1
299
+ - type: dot_recall@3
300
+ value: 0.31376984126984125
301
+ name: Dot Recall@3
302
+ - type: dot_recall@5
303
+ value: 0.35210317460317453
304
+ name: Dot Recall@5
305
+ - type: dot_recall@10
306
+ value: 0.42468253968253966
307
+ name: Dot Recall@10
308
+ - type: dot_ndcg@10
309
+ value: 0.3556599197720009
310
+ name: Dot Ndcg@10
311
+ - type: dot_mrr@10
312
+ value: 0.4181904761904762
313
+ name: Dot Mrr@10
314
+ - type: dot_map@100
315
+ value: 0.3060313184828012
316
+ name: Dot Map@100
317
+ - task:
318
+ type: sparse-information-retrieval
319
+ name: Sparse Information Retrieval
320
+ dataset:
321
+ name: NanoHotpotQA
322
+ type: NanoHotpotQA
323
+ metrics:
324
+ - type: dot_accuracy@1
325
+ value: 0.72
326
+ name: Dot Accuracy@1
327
+ - type: dot_accuracy@3
328
+ value: 0.86
329
+ name: Dot Accuracy@3
330
+ - type: dot_accuracy@5
331
+ value: 0.9
332
+ name: Dot Accuracy@5
333
+ - type: dot_accuracy@10
334
+ value: 0.94
335
+ name: Dot Accuracy@10
336
+ - type: dot_precision@1
337
+ value: 0.72
338
+ name: Dot Precision@1
339
+ - type: dot_precision@3
340
+ value: 0.4266666666666666
341
+ name: Dot Precision@3
342
+ - type: dot_precision@5
343
+ value: 0.2799999999999999
344
+ name: Dot Precision@5
345
+ - type: dot_precision@10
346
+ value: 0.14999999999999997
347
+ name: Dot Precision@10
348
+ - type: dot_recall@1
349
+ value: 0.36
350
+ name: Dot Recall@1
351
+ - type: dot_recall@3
352
+ value: 0.64
353
+ name: Dot Recall@3
354
+ - type: dot_recall@5
355
+ value: 0.7
356
+ name: Dot Recall@5
357
+ - type: dot_recall@10
358
+ value: 0.75
359
+ name: Dot Recall@10
360
+ - type: dot_ndcg@10
361
+ value: 0.6950779198152243
362
+ name: Dot Ndcg@10
363
+ - type: dot_mrr@10
364
+ value: 0.7948333333333334
365
+ name: Dot Mrr@10
366
+ - type: dot_map@100
367
+ value: 0.6244374457422245
368
+ name: Dot Map@100
369
+ - task:
370
+ type: sparse-information-retrieval
371
+ name: Sparse Information Retrieval
372
+ dataset:
373
+ name: NanoMSMARCO
374
+ type: NanoMSMARCO
375
+ metrics:
376
+ - type: dot_accuracy@1
377
+ value: 0.24
378
+ name: Dot Accuracy@1
379
+ - type: dot_accuracy@3
380
+ value: 0.52
381
+ name: Dot Accuracy@3
382
+ - type: dot_accuracy@5
383
+ value: 0.62
384
+ name: Dot Accuracy@5
385
+ - type: dot_accuracy@10
386
+ value: 0.74
387
+ name: Dot Accuracy@10
388
+ - type: dot_precision@1
389
+ value: 0.24
390
+ name: Dot Precision@1
391
+ - type: dot_precision@3
392
+ value: 0.1733333333333333
393
+ name: Dot Precision@3
394
+ - type: dot_precision@5
395
+ value: 0.124
396
+ name: Dot Precision@5
397
+ - type: dot_precision@10
398
+ value: 0.07400000000000001
399
+ name: Dot Precision@10
400
+ - type: dot_recall@1
401
+ value: 0.24
402
+ name: Dot Recall@1
403
+ - type: dot_recall@3
404
+ value: 0.52
405
+ name: Dot Recall@3
406
+ - type: dot_recall@5
407
+ value: 0.62
408
+ name: Dot Recall@5
409
+ - type: dot_recall@10
410
+ value: 0.74
411
+ name: Dot Recall@10
412
+ - type: dot_ndcg@10
413
+ value: 0.4858300241520006
414
+ name: Dot Ndcg@10
415
+ - type: dot_mrr@10
416
+ value: 0.40521428571428575
417
+ name: Dot Mrr@10
418
+ - type: dot_map@100
419
+ value: 0.42175748899886834
420
+ name: Dot Map@100
421
+ - task:
422
+ type: sparse-information-retrieval
423
+ name: Sparse Information Retrieval
424
+ dataset:
425
+ name: NanoNFCorpus
426
+ type: NanoNFCorpus
427
+ metrics:
428
+ - type: dot_accuracy@1
429
+ value: 0.42
430
+ name: Dot Accuracy@1
431
+ - type: dot_accuracy@3
432
+ value: 0.6
433
+ name: Dot Accuracy@3
434
+ - type: dot_accuracy@5
435
+ value: 0.66
436
+ name: Dot Accuracy@5
437
+ - type: dot_accuracy@10
438
+ value: 0.76
439
+ name: Dot Accuracy@10
440
+ - type: dot_precision@1
441
+ value: 0.42
442
+ name: Dot Precision@1
443
+ - type: dot_precision@3
444
+ value: 0.3466666666666666
445
+ name: Dot Precision@3
446
+ - type: dot_precision@5
447
+ value: 0.32799999999999996
448
+ name: Dot Precision@5
449
+ - type: dot_precision@10
450
+ value: 0.28200000000000003
451
+ name: Dot Precision@10
452
+ - type: dot_recall@1
453
+ value: 0.046687705999640026
454
+ name: Dot Recall@1
455
+ - type: dot_recall@3
456
+ value: 0.09790588476953502
457
+ name: Dot Recall@3
458
+ - type: dot_recall@5
459
+ value: 0.12000426530930396
460
+ name: Dot Recall@5
461
+ - type: dot_recall@10
462
+ value: 0.16155008782514965
463
+ name: Dot Recall@10
464
+ - type: dot_ndcg@10
465
+ value: 0.3519230892919392
466
+ name: Dot Ndcg@10
467
+ - type: dot_mrr@10
468
+ value: 0.5292380952380953
469
+ name: Dot Mrr@10
470
+ - type: dot_map@100
471
+ value: 0.16799707461195798
472
+ name: Dot Map@100
473
+ - task:
474
+ type: sparse-information-retrieval
475
+ name: Sparse Information Retrieval
476
+ dataset:
477
+ name: NanoNQ
478
+ type: NanoNQ
479
+ metrics:
480
+ - type: dot_accuracy@1
481
+ value: 0.44
482
+ name: Dot Accuracy@1
483
+ - type: dot_accuracy@3
484
+ value: 0.6
485
+ name: Dot Accuracy@3
486
+ - type: dot_accuracy@5
487
+ value: 0.68
488
+ name: Dot Accuracy@5
489
+ - type: dot_accuracy@10
490
+ value: 0.82
491
+ name: Dot Accuracy@10
492
+ - type: dot_precision@1
493
+ value: 0.44
494
+ name: Dot Precision@1
495
+ - type: dot_precision@3
496
+ value: 0.2
497
+ name: Dot Precision@3
498
+ - type: dot_precision@5
499
+ value: 0.14400000000000002
500
+ name: Dot Precision@5
501
+ - type: dot_precision@10
502
+ value: 0.09
503
+ name: Dot Precision@10
504
+ - type: dot_recall@1
505
+ value: 0.42
506
+ name: Dot Recall@1
507
+ - type: dot_recall@3
508
+ value: 0.56
509
+ name: Dot Recall@3
510
+ - type: dot_recall@5
511
+ value: 0.65
512
+ name: Dot Recall@5
513
+ - type: dot_recall@10
514
+ value: 0.79
515
+ name: Dot Recall@10
516
+ - type: dot_ndcg@10
517
+ value: 0.5968041069603208
518
+ name: Dot Ndcg@10
519
+ - type: dot_mrr@10
520
+ value: 0.5449682539682539
521
+ name: Dot Mrr@10
522
+ - type: dot_map@100
523
+ value: 0.5360944900687548
524
+ name: Dot Map@100
525
+ - task:
526
+ type: sparse-information-retrieval
527
+ name: Sparse Information Retrieval
528
+ dataset:
529
+ name: NanoQuoraRetrieval
530
+ type: NanoQuoraRetrieval
531
+ metrics:
532
+ - type: dot_accuracy@1
533
+ value: 0.8
534
+ name: Dot Accuracy@1
535
+ - type: dot_accuracy@3
536
+ value: 0.94
537
+ name: Dot Accuracy@3
538
+ - type: dot_accuracy@5
539
+ value: 0.98
540
+ name: Dot Accuracy@5
541
+ - type: dot_accuracy@10
542
+ value: 1.0
543
+ name: Dot Accuracy@10
544
+ - type: dot_precision@1
545
+ value: 0.8
546
+ name: Dot Precision@1
547
+ - type: dot_precision@3
548
+ value: 0.37999999999999995
549
+ name: Dot Precision@3
550
+ - type: dot_precision@5
551
+ value: 0.24799999999999997
552
+ name: Dot Precision@5
553
+ - type: dot_precision@10
554
+ value: 0.13399999999999998
555
+ name: Dot Precision@10
556
+ - type: dot_recall@1
557
+ value: 0.6973333333333332
558
+ name: Dot Recall@1
559
+ - type: dot_recall@3
560
+ value: 0.8946666666666667
561
+ name: Dot Recall@3
562
+ - type: dot_recall@5
563
+ value: 0.946
564
+ name: Dot Recall@5
565
+ - type: dot_recall@10
566
+ value: 0.99
567
+ name: Dot Recall@10
568
+ - type: dot_ndcg@10
569
+ value: 0.8922979605477963
570
+ name: Dot Ndcg@10
571
+ - type: dot_mrr@10
572
+ value: 0.8785238095238094
573
+ name: Dot Mrr@10
574
+ - type: dot_map@100
575
+ value: 0.8493405677655678
576
+ name: Dot Map@100
577
+ - task:
578
+ type: sparse-information-retrieval
579
+ name: Sparse Information Retrieval
580
+ dataset:
581
+ name: NanoSCIDOCS
582
+ type: NanoSCIDOCS
583
+ metrics:
584
+ - type: dot_accuracy@1
585
+ value: 0.36
586
+ name: Dot Accuracy@1
587
+ - type: dot_accuracy@3
588
+ value: 0.62
589
+ name: Dot Accuracy@3
590
+ - type: dot_accuracy@5
591
+ value: 0.7
592
+ name: Dot Accuracy@5
593
+ - type: dot_accuracy@10
594
+ value: 0.76
595
+ name: Dot Accuracy@10
596
+ - type: dot_precision@1
597
+ value: 0.36
598
+ name: Dot Precision@1
599
+ - type: dot_precision@3
600
+ value: 0.26666666666666666
601
+ name: Dot Precision@3
602
+ - type: dot_precision@5
603
+ value: 0.22399999999999998
604
+ name: Dot Precision@5
605
+ - type: dot_precision@10
606
+ value: 0.16599999999999998
607
+ name: Dot Precision@10
608
+ - type: dot_recall@1
609
+ value: 0.074
610
+ name: Dot Recall@1
611
+ - type: dot_recall@3
612
+ value: 0.16666666666666669
613
+ name: Dot Recall@3
614
+ - type: dot_recall@5
615
+ value: 0.23166666666666663
616
+ name: Dot Recall@5
617
+ - type: dot_recall@10
618
+ value: 0.34266666666666656
619
+ name: Dot Recall@10
620
+ - type: dot_ndcg@10
621
+ value: 0.32123645548157265
622
+ name: Dot Ndcg@10
623
+ - type: dot_mrr@10
624
+ value: 0.5074126984126984
625
+ name: Dot Mrr@10
626
+ - type: dot_map@100
627
+ value: 0.23675914234249176
628
+ name: Dot Map@100
629
+ - task:
630
+ type: sparse-information-retrieval
631
+ name: Sparse Information Retrieval
632
+ dataset:
633
+ name: NanoArguAna
634
+ type: NanoArguAna
635
+ metrics:
636
+ - type: dot_accuracy@1
637
+ value: 0.14
638
+ name: Dot Accuracy@1
639
+ - type: dot_accuracy@3
640
+ value: 0.5
641
+ name: Dot Accuracy@3
642
+ - type: dot_accuracy@5
643
+ value: 0.6
644
+ name: Dot Accuracy@5
645
+ - type: dot_accuracy@10
646
+ value: 0.74
647
+ name: Dot Accuracy@10
648
+ - type: dot_precision@1
649
+ value: 0.14
650
+ name: Dot Precision@1
651
+ - type: dot_precision@3
652
+ value: 0.16666666666666663
653
+ name: Dot Precision@3
654
+ - type: dot_precision@5
655
+ value: 0.12
656
+ name: Dot Precision@5
657
+ - type: dot_precision@10
658
+ value: 0.07400000000000001
659
+ name: Dot Precision@10
660
+ - type: dot_recall@1
661
+ value: 0.14
662
+ name: Dot Recall@1
663
+ - type: dot_recall@3
664
+ value: 0.5
665
+ name: Dot Recall@3
666
+ - type: dot_recall@5
667
+ value: 0.6
668
+ name: Dot Recall@5
669
+ - type: dot_recall@10
670
+ value: 0.74
671
+ name: Dot Recall@10
672
+ - type: dot_ndcg@10
673
+ value: 0.4389511719056823
674
+ name: Dot Ndcg@10
675
+ - type: dot_mrr@10
676
+ value: 0.3431904761904762
677
+ name: Dot Mrr@10
678
+ - type: dot_map@100
679
+ value: 0.35224302854950346
680
+ name: Dot Map@100
681
+ - task:
682
+ type: sparse-information-retrieval
683
+ name: Sparse Information Retrieval
684
+ dataset:
685
+ name: NanoSciFact
686
+ type: NanoSciFact
687
+ metrics:
688
+ - type: dot_accuracy@1
689
+ value: 0.48
690
+ name: Dot Accuracy@1
691
+ - type: dot_accuracy@3
692
+ value: 0.68
693
+ name: Dot Accuracy@3
694
+ - type: dot_accuracy@5
695
+ value: 0.7
696
+ name: Dot Accuracy@5
697
+ - type: dot_accuracy@10
698
+ value: 0.76
699
+ name: Dot Accuracy@10
700
+ - type: dot_precision@1
701
+ value: 0.48
702
+ name: Dot Precision@1
703
+ - type: dot_precision@3
704
+ value: 0.2333333333333333
705
+ name: Dot Precision@3
706
+ - type: dot_precision@5
707
+ value: 0.14800000000000002
708
+ name: Dot Precision@5
709
+ - type: dot_precision@10
710
+ value: 0.08599999999999998
711
+ name: Dot Precision@10
712
+ - type: dot_recall@1
713
+ value: 0.455
714
+ name: Dot Recall@1
715
+ - type: dot_recall@3
716
+ value: 0.65
717
+ name: Dot Recall@3
718
+ - type: dot_recall@5
719
+ value: 0.68
720
+ name: Dot Recall@5
721
+ - type: dot_recall@10
722
+ value: 0.76
723
+ name: Dot Recall@10
724
+ - type: dot_ndcg@10
725
+ value: 0.61443063378869
726
+ name: Dot Ndcg@10
727
+ - type: dot_mrr@10
728
+ value: 0.5731666666666667
729
+ name: Dot Mrr@10
730
+ - type: dot_map@100
731
+ value: 0.5696919873212444
732
+ name: Dot Map@100
733
+ - task:
734
+ type: sparse-information-retrieval
735
+ name: Sparse Information Retrieval
736
+ dataset:
737
+ name: NanoTouche2020
738
+ type: NanoTouche2020
739
+ metrics:
740
+ - type: dot_accuracy@1
741
+ value: 0.6326530612244898
742
+ name: Dot Accuracy@1
743
+ - type: dot_accuracy@3
744
+ value: 0.7959183673469388
745
+ name: Dot Accuracy@3
746
+ - type: dot_accuracy@5
747
+ value: 0.8367346938775511
748
+ name: Dot Accuracy@5
749
+ - type: dot_accuracy@10
750
+ value: 0.9591836734693877
751
+ name: Dot Accuracy@10
752
+ - type: dot_precision@1
753
+ value: 0.6326530612244898
754
+ name: Dot Precision@1
755
+ - type: dot_precision@3
756
+ value: 0.5374149659863945
757
+ name: Dot Precision@3
758
+ - type: dot_precision@5
759
+ value: 0.5020408163265306
760
+ name: Dot Precision@5
761
+ - type: dot_precision@10
762
+ value: 0.4326530612244897
763
+ name: Dot Precision@10
764
+ - type: dot_recall@1
765
+ value: 0.043721411012674946
766
+ name: Dot Recall@1
767
+ - type: dot_recall@3
768
+ value: 0.11111388641462987
769
+ name: Dot Recall@3
770
+ - type: dot_recall@5
771
+ value: 0.1725353206760411
772
+ name: Dot Recall@5
773
+ - type: dot_recall@10
774
+ value: 0.28394925382833736
775
+ name: Dot Recall@10
776
+ - type: dot_ndcg@10
777
+ value: 0.4877365323610393
778
+ name: Dot Ndcg@10
779
+ - type: dot_mrr@10
780
+ value: 0.7339002267573695
781
+ name: Dot Mrr@10
782
+ - type: dot_map@100
783
+ value: 0.3590109302813293
784
+ name: Dot Map@100
785
+ - task:
786
+ type: sparse-nano-beir
787
+ name: Sparse Nano BEIR
788
+ dataset:
789
+ name: NanoBEIR mean
790
+ type: NanoBEIR_mean
791
+ metrics:
792
+ - type: dot_accuracy@1
793
+ value: 0.46866562009419144
794
+ name: Dot Accuracy@1
795
+ - type: dot_accuracy@3
796
+ value: 0.6627629513343798
797
+ name: Dot Accuracy@3
798
+ - type: dot_accuracy@5
799
+ value: 0.7305180533751963
800
+ name: Dot Accuracy@5
801
+ - type: dot_accuracy@10
802
+ value: 0.8137833594976451
803
+ name: Dot Accuracy@10
804
+ - type: dot_precision@1
805
+ value: 0.46866562009419144
806
+ name: Dot Precision@1
807
+ - type: dot_precision@3
808
+ value: 0.29928833071690214
809
+ name: Dot Precision@3
810
+ - type: dot_precision@5
811
+ value: 0.23861852433281003
812
+ name: Dot Precision@5
813
+ - type: dot_precision@10
814
+ value: 0.1706656200941915
815
+ name: Dot Precision@10
816
+ - type: dot_recall@1
817
+ value: 0.27301664240337103
818
+ name: Dot Recall@1
819
+ - type: dot_recall@3
820
+ value: 0.4299467909817038
821
+ name: Dot Recall@3
822
+ - type: dot_recall@5
823
+ value: 0.4935344251180747
824
+ name: Dot Recall@5
825
+ - type: dot_recall@10
826
+ value: 0.5788989922903304
827
+ name: Dot Recall@10
828
+ - type: dot_ndcg@10
829
+ value: 0.5271621816528863
830
+ name: Dot Ndcg@10
831
+ - type: dot_mrr@10
832
+ value: 0.5802646084074655
833
+ name: Dot Mrr@10
834
+ - type: dot_map@100
835
+ value: 0.447532377602445
836
+ name: Dot Map@100
837
+ ---
838
+
839
+ # splade-cocondenser trained on GooAQ
840
+
841
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [Luyu/co-condenser-marco](https://huggingface.co/Luyu/co-condenser-marco) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
842
+
843
+ ## Model Details
844
+
845
+ ### Model Description
846
+ - **Model Type:** SPLADE Sparse Encoder
847
+ - **Base model:** [Luyu/co-condenser-marco](https://huggingface.co/Luyu/co-condenser-marco) <!-- at revision e0cef0ab2410aae0f0994366ddefb5649a266709 -->
848
+ - **Maximum Sequence Length:** 256 tokens
849
+ - **Output Dimensionality:** 30522 dimensions
850
+ - **Similarity Function:** Dot Product
851
+ - **Training Dataset:**
852
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
853
+ - **Language:** en
854
+ - **License:** apache-2.0
855
+
856
+ ### Model Sources
857
+
858
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
859
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
860
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
861
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
862
+
863
+ ### Full Model Architecture
864
+
865
+ ```
866
+ SparseEncoder(
867
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
868
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
869
+ )
870
+ ```
871
+
872
+ ## Usage
873
+
874
+ ### Direct Usage (Sentence Transformers)
875
+
876
+ First install the Sentence Transformers library:
877
+
878
+ ```bash
879
+ pip install -U sentence-transformers
880
+ ```
881
+
882
+ Then you can load this model and run inference.
883
+ ```python
884
+ from sentence_transformers import SparseEncoder
885
+
886
+ # Download from the 🤗 Hub
887
+ model = SparseEncoder("tomaarsen/splade-cocondenser-gooaq")
888
+ # Run inference
889
+ sentences = [
890
+ 'what is the difference between 18 and 20 inch tires?',
891
+ 'The only real difference is a 20" rim would be more likely to be damaged, as you pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the availability of tires will likely be much more limited for the larger rim. ... Tire selection is better for 18" wheels than 20" wheels.',
892
+ 'So extracurricular activities are just activities that you do outside of class. The Common App says that extracurricular activities "include arts, athletics, clubs, employment, personal commitments, and other pursuits."',
893
+ ]
894
+ embeddings = model.encode(sentences)
895
+ print(embeddings.shape)
896
+ # (3, 30522)
897
+
898
+ # Get the similarity scores for the embeddings
899
+ similarities = model.similarity(embeddings, embeddings)
900
+ print(similarities.shape)
901
+ # [3, 3]
902
+ ```
903
+
904
+ <!--
905
+ ### Direct Usage (Transformers)
906
+
907
+ <details><summary>Click to see the direct usage in Transformers</summary>
908
+
909
+ </details>
910
+ -->
911
+
912
+ <!--
913
+ ### Downstream Usage (Sentence Transformers)
914
+
915
+ You can finetune this model on your own dataset.
916
+
917
+ <details><summary>Click to expand</summary>
918
+
919
+ </details>
920
+ -->
921
+
922
+ <!--
923
+ ### Out-of-Scope Use
924
+
925
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
926
+ -->
927
+
928
+ ## Evaluation
929
+
930
+ ### Metrics
931
+
932
+ #### Sparse Information Retrieval
933
+
934
+ * Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
935
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
936
+
937
+ | Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
938
+ |:-----------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------|
939
+ | dot_accuracy@1 | 0.18 | 0.6 | 0.74 | 0.34 | 0.72 | 0.24 | 0.42 | 0.44 | 0.8 | 0.36 | 0.14 | 0.48 | 0.6327 |
940
+ | dot_accuracy@3 | 0.38 | 0.78 | 0.88 | 0.46 | 0.86 | 0.52 | 0.6 | 0.6 | 0.94 | 0.62 | 0.5 | 0.68 | 0.7959 |
941
+ | dot_accuracy@5 | 0.52 | 0.84 | 0.94 | 0.52 | 0.9 | 0.62 | 0.66 | 0.68 | 0.98 | 0.7 | 0.6 | 0.7 | 0.8367 |
942
+ | dot_accuracy@10 | 0.62 | 0.92 | 0.94 | 0.62 | 0.94 | 0.74 | 0.76 | 0.82 | 1.0 | 0.76 | 0.74 | 0.76 | 0.9592 |
943
+ | dot_precision@1 | 0.18 | 0.6 | 0.74 | 0.34 | 0.72 | 0.24 | 0.42 | 0.44 | 0.8 | 0.36 | 0.14 | 0.48 | 0.6327 |
944
+ | dot_precision@3 | 0.14 | 0.52 | 0.2933 | 0.2067 | 0.4267 | 0.1733 | 0.3467 | 0.2 | 0.38 | 0.2667 | 0.1667 | 0.2333 | 0.5374 |
945
+ | dot_precision@5 | 0.12 | 0.512 | 0.2 | 0.152 | 0.28 | 0.124 | 0.328 | 0.144 | 0.248 | 0.224 | 0.12 | 0.148 | 0.502 |
946
+ | dot_precision@10 | 0.08 | 0.452 | 0.102 | 0.096 | 0.15 | 0.074 | 0.282 | 0.09 | 0.134 | 0.166 | 0.074 | 0.086 | 0.4327 |
947
+ | dot_recall@1 | 0.115 | 0.0552 | 0.7167 | 0.1856 | 0.36 | 0.24 | 0.0467 | 0.42 | 0.6973 | 0.074 | 0.14 | 0.455 | 0.0437 |
948
+ | dot_recall@3 | 0.1983 | 0.1102 | 0.8267 | 0.3138 | 0.64 | 0.52 | 0.0979 | 0.56 | 0.8947 | 0.1667 | 0.5 | 0.65 | 0.1111 |
949
+ | dot_recall@5 | 0.2683 | 0.1586 | 0.9167 | 0.3521 | 0.7 | 0.62 | 0.12 | 0.65 | 0.946 | 0.2317 | 0.6 | 0.68 | 0.1725 |
950
+ | dot_recall@10 | 0.3323 | 0.2872 | 0.9233 | 0.4247 | 0.75 | 0.74 | 0.1616 | 0.79 | 0.99 | 0.3427 | 0.74 | 0.76 | 0.2839 |
951
+ | **dot_ndcg@10** | **0.2594** | **0.5254** | **0.8284** | **0.3557** | **0.6951** | **0.4858** | **0.3519** | **0.5968** | **0.8923** | **0.3212** | **0.439** | **0.6144** | **0.4877** |
952
+ | dot_mrr@10 | 0.3062 | 0.6989 | 0.8097 | 0.4182 | 0.7948 | 0.4052 | 0.5292 | 0.545 | 0.8785 | 0.5074 | 0.3432 | 0.5732 | 0.7339 |
953
+ | dot_map@100 | 0.2072 | 0.394 | 0.7934 | 0.306 | 0.6244 | 0.4218 | 0.168 | 0.5361 | 0.8493 | 0.2368 | 0.3522 | 0.5697 | 0.359 |
954
+
955
+ #### Sparse Nano BEIR
956
+
957
+ * Dataset: `NanoBEIR_mean`
958
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
959
+ ```json
960
+ {
961
+ "dataset_names": [
962
+ "climatefever",
963
+ "dbpedia",
964
+ "fever",
965
+ "fiqa2018",
966
+ "hotpotqa",
967
+ "msmarco",
968
+ "nfcorpus",
969
+ "nq",
970
+ "quoraretrieval",
971
+ "scidocs",
972
+ "arguana",
973
+ "scifact",
974
+ "touche2020"
975
+ ]
976
+ }
977
+ ```
978
+
979
+ | Metric | Value |
980
+ |:-----------------|:-----------|
981
+ | dot_accuracy@1 | 0.4687 |
982
+ | dot_accuracy@3 | 0.6628 |
983
+ | dot_accuracy@5 | 0.7305 |
984
+ | dot_accuracy@10 | 0.8138 |
985
+ | dot_precision@1 | 0.4687 |
986
+ | dot_precision@3 | 0.2993 |
987
+ | dot_precision@5 | 0.2386 |
988
+ | dot_precision@10 | 0.1707 |
989
+ | dot_recall@1 | 0.273 |
990
+ | dot_recall@3 | 0.4299 |
991
+ | dot_recall@5 | 0.4935 |
992
+ | dot_recall@10 | 0.5789 |
993
+ | **dot_ndcg@10** | **0.5272** |
994
+ | dot_mrr@10 | 0.5803 |
995
+ | dot_map@100 | 0.4475 |
996
+
997
+ <!--
998
+ ## Bias, Risks and Limitations
999
+
1000
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1001
+ -->
1002
+
1003
+ <!--
1004
+ ### Recommendations
1005
+
1006
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1007
+ -->
1008
+
1009
+ ## Training Details
1010
+
1011
+ ### Training Dataset
1012
+
1013
+ #### gooaq
1014
+
1015
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1016
+ * Size: 3,011,496 training samples
1017
+ * Columns: <code>question</code> and <code>answer</code>
1018
+ * Approximate statistics based on the first 1000 samples:
1019
+ | | question | answer |
1020
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1021
+ | type | string | string |
1022
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.87 tokens</li><li>max: 23 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.09 tokens</li><li>max: 201 tokens</li></ul> |
1023
+ * Samples:
1024
+ | question | answer |
1025
+ |:-----------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1026
+ | <code>what is the difference between clay and mud mask?</code> | <code>The main difference between the two is that mud is a skin-healing agent, while clay is a cosmetic, drying agent. Clay masks are most useful for someone who has oily skin and is prone to breakouts of acne and blemishes.</code> |
1027
+ | <code>myki how much on card?</code> | <code>A full fare myki card costs $6 and a concession, seniors or child myki costs $3. For more information about how to use your myki, visit ptv.vic.gov.au or call 1800 800 007.</code> |
1028
+ | <code>how to find out if someone blocked your phone number on iphone?</code> | <code>If you get a notification like "Message Not Delivered" or you get no notification at all, that's a sign of a potential block. Next, you could try calling the person. If the call goes right to voicemail or rings once (or a half ring) then goes to voicemail, that's further evidence you may have been blocked.</code> |
1029
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1030
+ ```json
1031
+ {'loss': SparseMultipleNegativesRankingLoss(
1032
+ (model): SparseEncoder(
1033
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1034
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1035
+ )
1036
+ (cross_entropy_loss): CrossEntropyLoss()
1037
+ ), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
1038
+ (model): SparseEncoder(
1039
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1040
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1041
+ )
1042
+ ), 'query_regularizer': FlopsLoss(
1043
+ (model): SparseEncoder(
1044
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1045
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1046
+ )
1047
+ )}
1048
+ ```
1049
+
1050
+ ### Evaluation Dataset
1051
+
1052
+ #### gooaq
1053
+
1054
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1055
+ * Size: 1,000 evaluation samples
1056
+ * Columns: <code>question</code> and <code>answer</code>
1057
+ * Approximate statistics based on the first 1000 samples:
1058
+ | | question | answer |
1059
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1060
+ | type | string | string |
1061
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.88 tokens</li><li>max: 22 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 61.03 tokens</li><li>max: 127 tokens</li></ul> |
1062
+ * Samples:
1063
+ | question | answer |
1064
+ |:-----------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1065
+ | <code>how do i program my directv remote with my tv?</code> | <code>['Press MENU on your remote.', 'Select Settings & Help > Settings > Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete programming.']</code> |
1066
+ | <code>are rodrigues fruit bats nocturnal?</code> | <code>Before its numbers were threatened by habitat destruction, storms, and hunting, some of those groups could number 500 or more members. Sunrise, sunset. Rodrigues fruit bats are most active at dawn, at dusk, and at night.</code> |
1067
+ | <code>why does your heart rate increase during exercise bbc bitesize?</code> | <code>During exercise there is an increase in physical activity and muscle cells respire more than they do when the body is at rest. The heart rate increases during exercise. The rate and depth of breathing increases - this makes sure that more oxygen is absorbed into the blood, and more carbon dioxide is removed from it.</code> |
1068
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1069
+ ```json
1070
+ {'loss': SparseMultipleNegativesRankingLoss(
1071
+ (model): SparseEncoder(
1072
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1073
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1074
+ )
1075
+ (cross_entropy_loss): CrossEntropyLoss()
1076
+ ), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
1077
+ (model): SparseEncoder(
1078
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1079
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1080
+ )
1081
+ ), 'query_regularizer': FlopsLoss(
1082
+ (model): SparseEncoder(
1083
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
1084
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1085
+ )
1086
+ )}
1087
+ ```
1088
+
1089
+ ### Training Hyperparameters
1090
+ #### Non-Default Hyperparameters
1091
+
1092
+ - `eval_strategy`: steps
1093
+ - `per_device_train_batch_size`: 16
1094
+ - `per_device_eval_batch_size`: 16
1095
+ - `learning_rate`: 2e-05
1096
+ - `num_train_epochs`: 1
1097
+ - `bf16`: True
1098
+ - `load_best_model_at_end`: True
1099
+ - `batch_sampler`: no_duplicates
1100
+
1101
+ #### All Hyperparameters
1102
+ <details><summary>Click to expand</summary>
1103
+
1104
+ - `overwrite_output_dir`: False
1105
+ - `do_predict`: False
1106
+ - `eval_strategy`: steps
1107
+ - `prediction_loss_only`: True
1108
+ - `per_device_train_batch_size`: 16
1109
+ - `per_device_eval_batch_size`: 16
1110
+ - `per_gpu_train_batch_size`: None
1111
+ - `per_gpu_eval_batch_size`: None
1112
+ - `gradient_accumulation_steps`: 1
1113
+ - `eval_accumulation_steps`: None
1114
+ - `torch_empty_cache_steps`: None
1115
+ - `learning_rate`: 2e-05
1116
+ - `weight_decay`: 0.0
1117
+ - `adam_beta1`: 0.9
1118
+ - `adam_beta2`: 0.999
1119
+ - `adam_epsilon`: 1e-08
1120
+ - `max_grad_norm`: 1.0
1121
+ - `num_train_epochs`: 1
1122
+ - `max_steps`: -1
1123
+ - `lr_scheduler_type`: linear
1124
+ - `lr_scheduler_kwargs`: {}
1125
+ - `warmup_ratio`: 0.0
1126
+ - `warmup_steps`: 0
1127
+ - `log_level`: passive
1128
+ - `log_level_replica`: warning
1129
+ - `log_on_each_node`: True
1130
+ - `logging_nan_inf_filter`: True
1131
+ - `save_safetensors`: True
1132
+ - `save_on_each_node`: False
1133
+ - `save_only_model`: False
1134
+ - `restore_callback_states_from_checkpoint`: False
1135
+ - `no_cuda`: False
1136
+ - `use_cpu`: False
1137
+ - `use_mps_device`: False
1138
+ - `seed`: 42
1139
+ - `data_seed`: None
1140
+ - `jit_mode_eval`: False
1141
+ - `use_ipex`: False
1142
+ - `bf16`: True
1143
+ - `fp16`: False
1144
+ - `fp16_opt_level`: O1
1145
+ - `half_precision_backend`: auto
1146
+ - `bf16_full_eval`: False
1147
+ - `fp16_full_eval`: False
1148
+ - `tf32`: None
1149
+ - `local_rank`: 0
1150
+ - `ddp_backend`: None
1151
+ - `tpu_num_cores`: None
1152
+ - `tpu_metrics_debug`: False
1153
+ - `debug`: []
1154
+ - `dataloader_drop_last`: False
1155
+ - `dataloader_num_workers`: 0
1156
+ - `dataloader_prefetch_factor`: None
1157
+ - `past_index`: -1
1158
+ - `disable_tqdm`: False
1159
+ - `remove_unused_columns`: True
1160
+ - `label_names`: None
1161
+ - `load_best_model_at_end`: True
1162
+ - `ignore_data_skip`: False
1163
+ - `fsdp`: []
1164
+ - `fsdp_min_num_params`: 0
1165
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1166
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1167
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1168
+ - `deepspeed`: None
1169
+ - `label_smoothing_factor`: 0.0
1170
+ - `optim`: adamw_torch
1171
+ - `optim_args`: None
1172
+ - `adafactor`: False
1173
+ - `group_by_length`: False
1174
+ - `length_column_name`: length
1175
+ - `ddp_find_unused_parameters`: None
1176
+ - `ddp_bucket_cap_mb`: None
1177
+ - `ddp_broadcast_buffers`: False
1178
+ - `dataloader_pin_memory`: True
1179
+ - `dataloader_persistent_workers`: False
1180
+ - `skip_memory_metrics`: True
1181
+ - `use_legacy_prediction_loop`: False
1182
+ - `push_to_hub`: False
1183
+ - `resume_from_checkpoint`: None
1184
+ - `hub_model_id`: None
1185
+ - `hub_strategy`: every_save
1186
+ - `hub_private_repo`: None
1187
+ - `hub_always_push`: False
1188
+ - `gradient_checkpointing`: False
1189
+ - `gradient_checkpointing_kwargs`: None
1190
+ - `include_inputs_for_metrics`: False
1191
+ - `include_for_metrics`: []
1192
+ - `eval_do_concat_batches`: True
1193
+ - `fp16_backend`: auto
1194
+ - `push_to_hub_model_id`: None
1195
+ - `push_to_hub_organization`: None
1196
+ - `mp_parameters`:
1197
+ - `auto_find_batch_size`: False
1198
+ - `full_determinism`: False
1199
+ - `torchdynamo`: None
1200
+ - `ray_scope`: last
1201
+ - `ddp_timeout`: 1800
1202
+ - `torch_compile`: False
1203
+ - `torch_compile_backend`: None
1204
+ - `torch_compile_mode`: None
1205
+ - `dispatch_batches`: None
1206
+ - `split_batches`: None
1207
+ - `include_tokens_per_second`: False
1208
+ - `include_num_input_tokens_seen`: False
1209
+ - `neftune_noise_alpha`: None
1210
+ - `optim_target_modules`: None
1211
+ - `batch_eval_metrics`: False
1212
+ - `eval_on_start`: False
1213
+ - `use_liger_kernel`: False
1214
+ - `eval_use_gather_object`: False
1215
+ - `average_tokens_across_devices`: False
1216
+ - `prompts`: None
1217
+ - `batch_sampler`: no_duplicates
1218
+ - `multi_dataset_batch_sampler`: proportional
1219
+
1220
+ </details>
1221
+
1222
+ ### Training Logs
1223
+ | Epoch | Step | Training Loss | Validation Loss | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 |
1224
+ |:----------:|:----------:|:-------------:|:---------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:-----------------------:|:------------------------:|:------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|:-------------------------:|
1225
+ | 0.0213 | 4000 | 0.3968 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1226
+ | 0.0425 | 8000 | 0.054 | 0.0224 | 0.2847 | 0.5628 | 0.8027 | 0.3260 | 0.6627 | 0.5252 | 0.3028 | 0.5467 | 0.7301 | 0.2563 | 0.3150 | 0.5072 | 0.4771 | 0.4846 |
1227
+ | 0.0638 | 12000 | 0.0468 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1228
+ | 0.0850 | 16000 | 0.0394 | 0.0137 | 0.1908 | 0.5269 | 0.7778 | 0.3464 | 0.6510 | 0.5374 | 0.3086 | 0.5719 | 0.7901 | 0.2900 | 0.3661 | 0.5473 | 0.4839 | 0.4914 |
1229
+ | 0.1063 | 20000 | 0.035 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1230
+ | 0.1275 | 24000 | 0.0402 | 0.0142 | 0.1971 | 0.5098 | 0.6363 | 0.3715 | 0.6979 | 0.5442 | 0.3555 | 0.5223 | 0.7881 | 0.3008 | 0.3401 | 0.5963 | 0.4795 | 0.4877 |
1231
+ | 0.1488 | 28000 | 0.0286 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1232
+ | 0.1700 | 32000 | 0.0289 | 0.0209 | 0.2097 | 0.5169 | 0.7501 | 0.3622 | 0.6629 | 0.5151 | 0.3239 | 0.5322 | 0.8189 | 0.3121 | 0.3045 | 0.5318 | 0.4748 | 0.4858 |
1233
+ | 0.1913 | 36000 | 0.0241 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1234
+ | 0.2125 | 40000 | 0.0243 | 0.0166 | 0.2150 | 0.4990 | 0.6614 | 0.3184 | 0.6564 | 0.5499 | 0.2924 | 0.5506 | 0.8177 | 0.2755 | 0.3214 | 0.5292 | 0.4605 | 0.4729 |
1235
+ | 0.2338 | 44000 | 0.021 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1236
+ | 0.2550 | 48000 | 0.0205 | 0.0045 | 0.2210 | 0.5328 | 0.5836 | 0.3180 | 0.6990 | 0.5365 | 0.2860 | 0.5529 | 0.8704 | 0.2860 | 0.4025 | 0.6107 | 0.4314 | 0.4870 |
1237
+ | 0.2763 | 52000 | 0.0181 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1238
+ | 0.2975 | 56000 | 0.018 | 0.0129 | 0.2131 | 0.5543 | 0.7181 | 0.3645 | 0.6852 | 0.5199 | 0.3232 | 0.5970 | 0.8914 | 0.2980 | 0.4618 | 0.5037 | 0.4592 | 0.5069 |
1239
+ | 0.3188 | 60000 | 0.0176 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1240
+ | 0.3400 | 64000 | 0.018 | 0.0141 | 0.2607 | 0.4594 | 0.7357 | 0.3597 | 0.6538 | 0.5082 | 0.3070 | 0.4944 | 0.8569 | 0.3252 | 0.4125 | 0.5243 | 0.4489 | 0.4882 |
1241
+ | 0.3613 | 68000 | 0.016 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1242
+ | 0.3825 | 72000 | 0.0143 | 0.0082 | 0.2737 | 0.5459 | 0.7570 | 0.3845 | 0.6806 | 0.5035 | 0.3408 | 0.5338 | 0.8608 | 0.2888 | 0.3096 | 0.6163 | 0.4709 | 0.5051 |
1243
+ | 0.4038 | 76000 | 0.0148 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1244
+ | 0.4250 | 80000 | 0.0135 | 0.0211 | 0.2267 | 0.4964 | 0.7829 | 0.3579 | 0.6758 | 0.4954 | 0.3195 | 0.5164 | 0.8698 | 0.2745 | 0.3012 | 0.6260 | 0.4426 | 0.4912 |
1245
+ | 0.4463 | 84000 | 0.0132 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1246
+ | 0.4675 | 88000 | 0.012 | 0.0270 | 0.2442 | 0.5741 | 0.8005 | 0.3372 | 0.7019 | 0.5064 | 0.3109 | 0.6238 | 0.8988 | 0.2805 | 0.3875 | 0.5590 | 0.4396 | 0.5126 |
1247
+ | 0.4888 | 92000 | 0.0126 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1248
+ | 0.5100 | 96000 | 0.0127 | 0.0201 | 0.2948 | 0.5384 | 0.7822 | 0.3800 | 0.6947 | 0.5237 | 0.3674 | 0.5646 | 0.8843 | 0.2873 | 0.3825 | 0.5898 | 0.4812 | 0.5208 |
1249
+ | 0.5313 | 100000 | 0.0113 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1250
+ | 0.5525 | 104000 | 0.0112 | 0.0057 | 0.2318 | 0.5091 | 0.8362 | 0.3649 | 0.6829 | 0.4695 | 0.3442 | 0.5403 | 0.8920 | 0.2696 | 0.3787 | 0.6109 | 0.4384 | 0.5053 |
1251
+ | 0.5738 | 108000 | 0.0094 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1252
+ | 0.5951 | 112000 | 0.0095 | 0.0101 | 0.2325 | 0.5184 | 0.7349 | 0.3672 | 0.6673 | 0.4474 | 0.3196 | 0.5647 | 0.8866 | 0.2938 | 0.3345 | 0.5744 | 0.4609 | 0.4925 |
1253
+ | 0.6163 | 116000 | 0.0096 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1254
+ | 0.6376 | 120000 | 0.01 | 0.0084 | 0.2362 | 0.4989 | 0.8299 | 0.3595 | 0.6820 | 0.5200 | 0.3286 | 0.6138 | 0.8959 | 0.3088 | 0.4139 | 0.5808 | 0.4833 | 0.5194 |
1255
+ | 0.6588 | 124000 | 0.0103 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1256
+ | 0.6801 | 128000 | 0.0082 | 0.0115 | 0.2402 | 0.5127 | 0.7943 | 0.3828 | 0.6796 | 0.4925 | 0.3337 | 0.5848 | 0.8956 | 0.2880 | 0.3962 | 0.5981 | 0.4634 | 0.5124 |
1257
+ | 0.7013 | 132000 | 0.0085 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1258
+ | 0.7226 | 136000 | 0.0087 | 0.0125 | 0.2444 | 0.5258 | 0.7659 | 0.3397 | 0.6939 | 0.4942 | 0.3330 | 0.5573 | 0.8866 | 0.2789 | 0.3829 | 0.5305 | 0.4699 | 0.5002 |
1259
+ | 0.7438 | 140000 | 0.0092 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1260
+ | 0.7651 | 144000 | 0.0084 | 0.0071 | 0.2376 | 0.5247 | 0.8359 | 0.3551 | 0.6987 | 0.4440 | 0.3230 | 0.5973 | 0.8875 | 0.3052 | 0.4243 | 0.5601 | 0.4865 | 0.5138 |
1261
+ | 0.7863 | 148000 | 0.0082 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1262
+ | 0.8076 | 152000 | 0.0073 | 0.0036 | 0.2379 | 0.5045 | 0.8240 | 0.3389 | 0.7027 | 0.4895 | 0.3373 | 0.5893 | 0.8878 | 0.2870 | 0.3998 | 0.5728 | 0.4735 | 0.5112 |
1263
+ | 0.8288 | 156000 | 0.0069 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1264
+ | **0.8501** | **160000** | **0.0076** | **0.0024** | **0.2594** | **0.5254** | **0.8284** | **0.3557** | **0.6951** | **0.4858** | **0.3519** | **0.5968** | **0.8923** | **0.3212** | **0.439** | **0.6144** | **0.4877** | **0.5272** |
1265
+ | 0.8713 | 164000 | 0.0062 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1266
+ | 0.8926 | 168000 | 0.0061 | 0.0084 | 0.2580 | 0.5068 | 0.8307 | 0.3629 | 0.7095 | 0.5132 | 0.3373 | 0.5577 | 0.8803 | 0.3041 | 0.4438 | 0.5802 | 0.4668 | 0.5193 |
1267
+ | 0.9138 | 172000 | 0.0067 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1268
+ | 0.9351 | 176000 | 0.0072 | 0.0076 | 0.2627 | 0.4988 | 0.8192 | 0.3587 | 0.7072 | 0.4968 | 0.3488 | 0.5746 | 0.8794 | 0.3049 | 0.4671 | 0.5872 | 0.4739 | 0.5215 |
1269
+ | 0.9563 | 180000 | 0.0049 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1270
+ | 0.9776 | 184000 | 0.0056 | 0.0067 | 0.2672 | 0.4954 | 0.8207 | 0.3473 | 0.7148 | 0.4997 | 0.3479 | 0.5798 | 0.8778 | 0.3115 | 0.4557 | 0.5884 | 0.4753 | 0.5216 |
1271
+ | 0.9988 | 188000 | 0.005 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1272
+ | -1 | -1 | - | - | 0.2594 | 0.5254 | 0.8284 | 0.3557 | 0.6951 | 0.4858 | 0.3519 | 0.5968 | 0.8923 | 0.3212 | 0.4390 | 0.6144 | 0.4877 | 0.5272 |
1273
+
1274
+ * The bold row denotes the saved checkpoint.
1275
+
1276
+ ### Environmental Impact
1277
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1278
+ - **Energy Consumed**: 2.656 kWh
1279
+ - **Carbon Emitted**: 1.032 kg of CO2
1280
+ - **Hours Used**: 9.368 hours
1281
+
1282
+ ### Training Hardware
1283
+ - **On Cloud**: No
1284
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1285
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1286
+ - **RAM Size**: 31.78 GB
1287
+
1288
+ ### Framework Versions
1289
+ - Python: 3.11.6
1290
+ - Sentence Transformers: 4.2.0.dev0
1291
+ - Transformers: 4.49.0
1292
+ - PyTorch: 2.6.0+cu124
1293
+ - Accelerate: 1.5.1
1294
+ - Datasets: 2.21.0
1295
+ - Tokenizers: 0.21.1
1296
+
1297
+ ## Citation
1298
+
1299
+ ### BibTeX
1300
+
1301
+ #### Sentence Transformers
1302
+ ```bibtex
1303
+ @inproceedings{reimers-2019-sentence-bert,
1304
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1305
+ author = "Reimers, Nils and Gurevych, Iryna",
1306
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1307
+ month = "11",
1308
+ year = "2019",
1309
+ publisher = "Association for Computational Linguistics",
1310
+ url = "https://arxiv.org/abs/1908.10084",
1311
+ }
1312
+ ```
1313
+
1314
+ #### SpladeLoss
1315
+ ```bibtex
1316
+ @misc{formal2022distillationhardnegativesampling,
1317
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1318
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1319
+ year={2022},
1320
+ eprint={2205.04733},
1321
+ archivePrefix={arXiv},
1322
+ primaryClass={cs.IR},
1323
+ url={https://arxiv.org/abs/2205.04733},
1324
+ }
1325
+ ```
1326
+
1327
+ <!--
1328
+ ## Glossary
1329
+
1330
+ *Clearly define terms in order to be accessible across audiences.*
1331
+ -->
1332
+
1333
+ <!--
1334
+ ## Model Card Authors
1335
+
1336
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1337
+ -->
1338
+
1339
+ <!--
1340
+ ## Model Card Contact
1341
+
1342
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1343
+ -->
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "Luyu/co-condenser-marco",
3
+ "architectures": [
4
+ "BertForMaskedLM"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.49.0",
23
+ "type_vocab_size": 2,
24
+ "use_cache": true,
25
+ "vocab_size": 30522
26
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.49.0",
6
+ "pytorch": "2.6.0+cu124"
7
+ },
8
+ "prompts": {},
9
+ "default_prompt_name": null,
10
+ "similarity_fn_name": "dot"
11
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fbb8e7990793bc122c0f134cb49015e2b01c5fe68827bc018ca11f17d16f87a7
3
+ size 438080896
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "extra_special_tokens": {},
49
+ "mask_token": "[MASK]",
50
+ "model_max_length": 512,
51
+ "never_split": null,
52
+ "pad_token": "[PAD]",
53
+ "sep_token": "[SEP]",
54
+ "strip_accents": null,
55
+ "tokenize_chinese_chars": true,
56
+ "tokenizer_class": "BertTokenizer",
57
+ "unk_token": "[UNK]"
58
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff