Instructions to use fyaronskiy/english_code_retriever with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fyaronskiy/english_code_retriever with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("fyaronskiy/english_code_retriever")

sentences = [
    "search_query: Finds the top long, short, and absolute positions.\n\n    Parameters\n    ----------\n    positions : pd.DataFrame\n        The positions that the strategy takes over time.\n    top : int, optional\n        How many of each to find (default 10).\n\n    Returns\n    -------\n    df_top_long : pd.DataFrame\n        Top long positions.\n    df_top_short : pd.DataFrame\n        Top short positions.\n    df_top_abs : pd.DataFrame\n        Top absolute positions.",
    "search_document: def symmetric_ema(xolds, yolds, low=None, high=None, n=512, decay_steps=1., low_counts_threshold=1e-8):\n    '''\n    perform symmetric EMA (exponential moving average)\n    smoothing and resampling to an even grid with n points.\n    Does not do extrapolation, so we assume\n    xolds[0] <= low && high <= xolds[-1]\n\n    Arguments:\n\n    xolds: array or list  - x values of data. Needs to be sorted in ascending order\n    yolds: array of list  - y values of data. Has to have the same length as xolds\n\n    low: float            - min value of the new x grid. By default equals to xolds[0]\n    high: float           - max value of the new x grid. By default equals to xolds[-1]\n\n    n: int                - number of points in new x grid\n\n    decay_steps: float    - EMA decay factor, expressed in new x grid steps.\n\n    low_counts_threshold: float or int\n                          - y values with counts less than this value will be set to NaN\n\n    Returns:\n        tuple sum_ys, count_ys where\n            xs        - array with new x grid\n            ys        - array of EMA of y at each point of the new x grid\n            count_ys  - array of EMA of y counts at each point of the new x grid\n\n    '''\n    xs, ys1, count_ys1 = one_sided_ema(xolds, yolds, low, high, n, decay_steps, low_counts_threshold=0)\n    _,  ys2, count_ys2 = one_sided_ema(-xolds[::-1], yolds[::-1], -high, -low, n, decay_steps, low_counts_threshold=0)\n    ys2 = ys2[::-1]\n    count_ys2 = count_ys2[::-1]\n    count_ys = count_ys1 + count_ys2\n    ys = (ys1 * count_ys1 + ys2 * count_ys2) / count_ys\n    ys[count_ys < low_counts_threshold] = np.nan\n    return xs, ys, count_ys",
    "search_document: def project(self, from_shape, to_shape):\n        \"\"\"\n        Project the polygon onto an image with different shape.\n\n        The relative coordinates of all points remain the same.\n        E.g. a point at (x=20, y=20) on an image (width=100, height=200) will be\n        projected on a new image (width=200, height=100) to (x=40, y=10).\n\n        This is intended for cases where the original image is resized.\n        It cannot be used for more complex changes (e.g. padding, cropping).\n\n        Parameters\n        ----------\n        from_shape : tuple of int\n            Shape of the original image. (Before resize.)\n\n        to_shape : tuple of int\n            Shape of the new image. (After resize.)\n\n        Returns\n        -------\n        imgaug.Polygon\n            Polygon object with new coordinates.\n\n        \"\"\"\n        if from_shape[0:2] == to_shape[0:2]:\n            return self.copy()\n        ls_proj = self.to_line_string(closed=False).project(\n            from_shape, to_shape)\n        return self.copy(exterior=ls_proj.coords)",
    "search_document: def get_top_long_short_abs(positions, top=10):\n    \"\"\"\n    Finds the top long, short, and absolute positions.\n\n    Parameters\n    ----------\n    positions : pd.DataFrame\n        The positions that the strategy takes over time.\n    top : int, optional\n        How many of each to find (default 10).\n\n    Returns\n    -------\n    df_top_long : pd.DataFrame\n        Top long positions.\n    df_top_short : pd.DataFrame\n        Top short positions.\n    df_top_abs : pd.DataFrame\n        Top absolute positions.\n    \"\"\"\n\n    positions = positions.drop('cash', axis='columns')\n    df_max = positions.max()\n    df_min = positions.min()\n    df_abs_max = positions.abs().max()\n    df_top_long = df_max[df_max > 0].nlargest(top)\n    df_top_short = df_min[df_min < 0].nsmallest(top)\n    df_top_abs = df_abs_max.nlargest(top)\n    return df_top_long, df_top_short, df_top_abs"
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

fyaronskiy commited on Oct 6, 2025

Commit

3ed636a

verified ·

1 Parent(s): 6d968fa

Add new SentenceTransformer model

Browse files

Files changed (10) hide show

1_Pooling/config.json +10 -0
README.md +1215 -0
config.json +45 -0
config_sentence_transformers.json +16 -0
model.safetensors +3 -0
modules.json +14 -0
sentence_bert_config.json +4 -0
special_tokens_map.json +37 -0
tokenizer.json +0 -0
tokenizer_config.json +952 -0

1_Pooling/config.json ADDED Viewed

	@@ -0,0 +1,10 @@

+{
+    "word_embedding_dimension": 768,
+    "pooling_mode_cls_token": false,
+    "pooling_mode_mean_tokens": true,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "include_prompt": true
+}

README.md ADDED Viewed

	@@ -0,0 +1,1215 @@

+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- dense
+- generated_from_trainer
+- dataset_size:1880853
+- loss:MultipleNegativesRankingLoss
+widget:
+- source_sentence: "search_query: Finds the top long, short, and absolute positions.\n\
+    \n    Parameters\n    ----------\n    positions : pd.DataFrame\n        The positions\
+    \ that the strategy takes over time.\n    top : int, optional\n        How many\
+    \ of each to find (default 10).\n\n    Returns\n    -------\n    df_top_long :\
+    \ pd.DataFrame\n        Top long positions.\n    df_top_short : pd.DataFrame\n\
+    \        Top short positions.\n    df_top_abs : pd.DataFrame\n        Top absolute\
+    \ positions."
+  sentences:
+  - "search_document: def symmetric_ema(xolds, yolds, low=None, high=None, n=512,\
+    \ decay_steps=1., low_counts_threshold=1e-8):\n    '''\n    perform symmetric\
+    \ EMA (exponential moving average)\n    smoothing and resampling to an even grid\
+    \ with n points.\n    Does not do extrapolation, so we assume\n    xolds[0] <=\
+    \ low && high <= xolds[-1]\n\n    Arguments:\n\n    xolds: array or list  - x\
+    \ values of data. Needs to be sorted in ascending order\n    yolds: array of list\
+    \  - y values of data. Has to have the same length as xolds\n\n    low: float\
+    \            - min value of the new x grid. By default equals to xolds[0]\n  \
+    \  high: float           - max value of the new x grid. By default equals to xolds[-1]\n\
+    \n    n: int                - number of points in new x grid\n\n    decay_steps:\
+    \ float    - EMA decay factor, expressed in new x grid steps.\n\n    low_counts_threshold:\
+    \ float or int\n                          - y values with counts less than this\
+    \ value will be set to NaN\n\n    Returns:\n        tuple sum_ys, count_ys where\n\
+    \            xs        - array with new x grid\n            ys        - array\
+    \ of EMA of y at each point of the new x grid\n            count_ys  - array of\
+    \ EMA of y counts at each point of the new x grid\n\n    '''\n    xs, ys1, count_ys1\
+    \ = one_sided_ema(xolds, yolds, low, high, n, decay_steps, low_counts_threshold=0)\n\
+    \    _,  ys2, count_ys2 = one_sided_ema(-xolds[::-1], yolds[::-1], -high, -low,\
+    \ n, decay_steps, low_counts_threshold=0)\n    ys2 = ys2[::-1]\n    count_ys2\
+    \ = count_ys2[::-1]\n    count_ys = count_ys1 + count_ys2\n    ys = (ys1 * count_ys1\
+    \ + ys2 * count_ys2) / count_ys\n    ys[count_ys < low_counts_threshold] = np.nan\n\
+    \    return xs, ys, count_ys"
+  - "search_document: def project(self, from_shape, to_shape):\n        \"\"\"\n \
+    \       Project the polygon onto an image with different shape.\n\n        The\
+    \ relative coordinates of all points remain the same.\n        E.g. a point at\
+    \ (x=20, y=20) on an image (width=100, height=200) will be\n        projected\
+    \ on a new image (width=200, height=100) to (x=40, y=10).\n\n        This is intended\
+    \ for cases where the original image is resized.\n        It cannot be used for\
+    \ more complex changes (e.g. padding, cropping).\n\n        Parameters\n     \
+    \   ----------\n        from_shape : tuple of int\n            Shape of the original\
+    \ image. (Before resize.)\n\n        to_shape : tuple of int\n            Shape\
+    \ of the new image. (After resize.)\n\n        Returns\n        -------\n    \
+    \    imgaug.Polygon\n            Polygon object with new coordinates.\n\n    \
+    \    \"\"\"\n        if from_shape[0:2] == to_shape[0:2]:\n            return\
+    \ self.copy()\n        ls_proj = self.to_line_string(closed=False).project(\n\
+    \            from_shape, to_shape)\n        return self.copy(exterior=ls_proj.coords)"
+  - "search_document: def get_top_long_short_abs(positions, top=10):\n    \"\"\"\n\
+    \    Finds the top long, short, and absolute positions.\n\n    Parameters\n  \
+    \  ----------\n    positions : pd.DataFrame\n        The positions that the strategy\
+    \ takes over time.\n    top : int, optional\n        How many of each to find\
+    \ (default 10).\n\n    Returns\n    -------\n    df_top_long : pd.DataFrame\n\
+    \        Top long positions.\n    df_top_short : pd.DataFrame\n        Top short\
+    \ positions.\n    df_top_abs : pd.DataFrame\n        Top absolute positions.\n\
+    \    \"\"\"\n\n    positions = positions.drop('cash', axis='columns')\n    df_max\
+    \ = positions.max()\n    df_min = positions.min()\n    df_abs_max = positions.abs().max()\n\
+    \    df_top_long = df_max[df_max > 0].nlargest(top)\n    df_top_short = df_min[df_min\
+    \ < 0].nsmallest(top)\n    df_top_abs = df_abs_max.nlargest(top)\n    return df_top_long,\
+    \ df_top_short, df_top_abs"
+- source_sentence: "search_query: Draw text on an image.\n\n    This uses by default\
+    \ DejaVuSans as its font, which is included in this library.\n\n    dtype support::\n\
+    \n        * ``uint8``: yes; fully tested\n        * ``uint16``: no\n        *\
+    \ ``uint32``: no\n        * ``uint64``: no\n        * ``int8``: no\n        *\
+    \ ``int16``: no\n        * ``int32``: no\n        * ``int64``: no\n        * ``float16``:\
+    \ no\n        * ``float32``: yes; not tested\n        * ``float64``: no\n    \
+    \    * ``float128``: no\n        * ``bool``: no\n\n        TODO check if other\
+    \ dtypes could be enabled\n\n    Parameters\n    ----------\n    img : (H,W,3)\
+    \ ndarray\n        The image array to draw text on.\n        Expected to be of\
+    \ dtype uint8 or float32 (value range 0.0 to 255.0).\n\n    y : int\n        x-coordinate\
+    \ of the top left corner of the text.\n\n    x : int\n        y- coordinate of\
+    \ the top left corner of the text.\n\n    text : str\n        The text to draw.\n\
+    \n    color : iterable of int, optional\n        Color of the text to draw. For\
+    \ RGB-images this is expected to be an RGB color.\n\n    size : int, optional\n\
+    \        Font size of the text to draw.\n\n    Returns\n    -------\n    img_np\
+    \ : (H,W,3) ndarray\n        Input image with text drawn on it."
+  sentences:
+  - "search_document: def cross_entropy_seq_with_mask(logits, target_seqs, input_mask,\
+    \ return_details=False, name=None):\n    \"\"\"Returns the expression of cross-entropy\
+    \ of two sequences, implement\n    softmax internally. Normally be used for Dynamic\
+    \ RNN with Synced sequence input and output.\n\n    Parameters\n    -----------\n\
+    \    logits : Tensor\n        2D tensor with shape of [batch_size * ?, n_classes],\
+    \ `?` means dynamic IDs for each example.\n        - Can be get from `DynamicRNNLayer`\
+    \ by setting ``return_seq_2d`` to `True`.\n    target_seqs : Tensor\n        int\
+    \ of tensor, like word ID. [batch_size, ?], `?` means dynamic IDs for each example.\n\
+    \    input_mask : Tensor\n        The mask to compute loss, it has the same size\
+    \ with `target_seqs`, normally 0 or 1.\n    return_details : boolean\n       \
+    \ Whether to return detailed losses.\n            - If False (default), only returns\
+    \ the loss.\n            - If True, returns the loss, losses, weights and targets\
+    \ (see source code).\n\n    Examples\n    --------\n    >>> batch_size = 64\n\
+    \    >>> vocab_size = 10000\n    >>> embedding_size = 256\n    >>> input_seqs\
+    \ = tf.placeholder(dtype=tf.int64, shape=[batch_size, None], name=\"input\")\n\
+    \    >>> target_seqs = tf.placeholder(dtype=tf.int64, shape=[batch_size, None],\
+    \ name=\"target\")\n    >>> input_mask = tf.placeholder(dtype=tf.int64, shape=[batch_size,\
+    \ None], name=\"mask\")\n    >>> net = tl.layers.EmbeddingInputlayer(\n    ...\
+    \         inputs = input_seqs,\n    ...         vocabulary_size = vocab_size,\n\
+    \    ...         embedding_size = embedding_size,\n    ...         name = 'seq_embedding')\n\
+    \    >>> net = tl.layers.DynamicRNNLayer(net,\n    ...         cell_fn = tf.contrib.rnn.BasicLSTMCell,\n\
+    \    ...         n_hidden = embedding_size,\n    ...         dropout = (0.7 if\
+    \ is_train else None),\n    ...         sequence_length = tl.layers.retrieve_seq_length_op2(input_seqs),\n\
+    \    ...         return_seq_2d = True,\n    ...         name = 'dynamicrnn')\n\
+    \    >>> print(net.outputs)\n    (?, 256)\n    >>> net = tl.layers.DenseLayer(net,\
+    \ n_units=vocab_size, name=\"output\")\n    >>> print(net.outputs)\n    (?, 10000)\n\
+    \    >>> loss = tl.cost.cross_entropy_seq_with_mask(net.outputs, target_seqs,\
+    \ input_mask)\n\n    \"\"\"\n    targets = tf.reshape(target_seqs, [-1])  # to\
+    \ one vector\n    weights = tf.to_float(tf.reshape(input_mask, [-1]))  # to one\
+    \ vector like targets\n    losses = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,\
+    \ labels=targets, name=name) * weights\n    # losses = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,\
+    \ labels=targets, name=name)) # for TF1.0 and others\n\n    loss = tf.divide(\n\
+    \        tf.reduce_sum(losses),  # loss from mask. reduce_sum before element-wise\
+    \ mul with mask !!\n        tf.reduce_sum(weights),\n        name=\"seq_loss_with_mask\"\
+    \n    )\n\n    if return_details:\n        return loss, losses, weights, targets\n\
+    \    else:\n        return loss"
+  - "search_document: def pickle_load(path, compression=False):\n    \"\"\"Unpickle\
+    \ a possible compressed pickle.\n\n    Parameters\n    ----------\n    path: str\n\
+    \        path to the output file\n    compression: bool\n        if true assumes\
+    \ that pickle was compressed when created and attempts decompression.\n\n    Returns\n\
+    \    -------\n    obj: object\n        the unpickled object\n    \"\"\"\n\n  \
+    \  if compression:\n        with zipfile.ZipFile(path, \"r\", compression=zipfile.ZIP_DEFLATED)\
+    \ as myzip:\n            with myzip.open(\"data\") as f:\n                return\
+    \ pickle.load(f)\n    else:\n        with open(path, \"rb\") as f:\n         \
+    \   return pickle.load(f)"
+  - "search_document: def draw_text(img, y, x, text, color=(0, 255, 0), size=25):\n\
+    \    \"\"\"\n    Draw text on an image.\n\n    This uses by default DejaVuSans\
+    \ as its font, which is included in this library.\n\n    dtype support::\n\n \
+    \       * ``uint8``: yes; fully tested\n        * ``uint16``: no\n        * ``uint32``:\
+    \ no\n        * ``uint64``: no\n        * ``int8``: no\n        * ``int16``: no\n\
+    \        * ``int32``: no\n        * ``int64``: no\n        * ``float16``: no\n\
+    \        * ``float32``: yes; not tested\n        * ``float64``: no\n        *\
+    \ ``float128``: no\n        * ``bool``: no\n\n        TODO check if other dtypes\
+    \ could be enabled\n\n    Parameters\n    ----------\n    img : (H,W,3) ndarray\n\
+    \        The image array to draw text on.\n        Expected to be of dtype uint8\
+    \ or float32 (value range 0.0 to 255.0).\n\n    y : int\n        x-coordinate\
+    \ of the top left corner of the text.\n\n    x : int\n        y- coordinate of\
+    \ the top left corner of the text.\n\n    text : str\n        The text to draw.\n\
+    \n    color : iterable of int, optional\n        Color of the text to draw. For\
+    \ RGB-images this is expected to be an RGB color.\n\n    size : int, optional\n\
+    \        Font size of the text to draw.\n\n    Returns\n    -------\n    img_np\
+    \ : (H,W,3) ndarray\n        Input image with text drawn on it.\n\n    \"\"\"\n\
+    \    do_assert(img.dtype in [np.uint8, np.float32])\n\n    input_dtype = img.dtype\n\
+    \    if img.dtype == np.float32:\n        img = img.astype(np.uint8)\n\n    img\
+    \ = PIL_Image.fromarray(img)\n    font = PIL_ImageFont.truetype(DEFAULT_FONT_FP,\
+    \ size)\n    context = PIL_ImageDraw.Draw(img)\n    context.text((x, y), text,\
+    \ fill=tuple(color), font=font)\n    img_np = np.asarray(img)\n\n    # PIL/asarray\
+    \ returns read only array\n    if not img_np.flags[\"WRITEABLE\"]:\n        try:\n\
+    \            # this seems to no longer work with np 1.16 (or was pillow updated?)\n\
+    \            img_np.setflags(write=True)\n        except ValueError as ex:\n \
+    \           if \"cannot set WRITEABLE flag to True of this array\" in str(ex):\n\
+    \                img_np = np.copy(img_np)\n\n    if img_np.dtype != input_dtype:\n\
+    \        img_np = img_np.astype(input_dtype)\n\n    return img_np"
+- source_sentence: "search_query: Choice and return an an action by given the action\
+    \ probability distribution.\n\n    Parameters\n    ------------\n    probs : list\
+    \ of float.\n        The probability distribution of all actions.\n    action_list\
+    \ : None or a list of int or others\n        A list of action in integer, string\
+    \ or others. If None, returns an integer range between 0 and len(probs)-1.\n\n\
+    \    Returns\n    --------\n    float int or str\n        The chosen action.\n\
+    \n    Examples\n    ----------\n    >>> for _ in range(5):\n    >>>     a = choice_action_by_probs([0.2,\
+    \ 0.4, 0.4])\n    >>>     print(a)\n    0\n    1\n    1\n    2\n    1\n    >>>\
+    \ for _ in range(3):\n    >>>     a = choice_action_by_probs([0.5, 0.5], ['a',\
+    \ 'b'])\n    >>>     print(a)\n    a\n    b\n    b"
+  sentences:
+  - "search_document: def from_keypoint_image(image, if_not_found_coords={\"x\": -1,\
+    \ \"y\": -1}, threshold=1, nb_channels=None): # pylint: disable=locally-disabled,\
+    \ dangerous-default-value, line-too-long\n        \"\"\"\n        Converts an\
+    \ image generated by ``to_keypoint_image()`` back to a KeypointsOnImage object.\n\
+    \n        Parameters\n        ----------\n        image : (H,W,N) ndarray\n  \
+    \          The keypoints image. N is the number of keypoints.\n\n        if_not_found_coords\
+    \ : tuple or list or dict or None, optional\n            Coordinates to use for\
+    \ keypoints that cannot be found in `image`.\n            If this is a list/tuple,\
+    \ it must have two integer values.\n            If it is a dictionary, it must\
+    \ have the keys ``x`` and ``y`` with\n            each containing one integer\
+    \ value.\n            If this is None, then the keypoint will not be added to\
+    \ the final\n            KeypointsOnImage object.\n\n        threshold : int,\
+    \ optional\n            The search for keypoints works by searching for the argmax\
+    \ in\n            each channel. This parameters contains the minimum value that\n\
+    \            the max must have in order to be viewed as a keypoint.\n\n      \
+    \  nb_channels : None or int, optional\n            Number of channels of the\
+    \ image on which the keypoints are placed.\n            Some keypoint augmenters\
+    \ require that information.\n            If set to None, the keypoint's shape\
+    \ will be set\n            to ``(height, width)``, otherwise ``(height, width,\
+    \ nb_channels)``.\n\n        Returns\n        -------\n        out : KeypointsOnImage\n\
+    \            The extracted keypoints.\n\n        \"\"\"\n        ia.do_assert(len(image.shape)\
+    \ == 3)\n        height, width, nb_keypoints = image.shape\n\n        drop_if_not_found\
+    \ = False\n        if if_not_found_coords is None:\n            drop_if_not_found\
+    \ = True\n            if_not_found_x = -1\n            if_not_found_y = -1\n \
+    \       elif isinstance(if_not_found_coords, (tuple, list)):\n            ia.do_assert(len(if_not_found_coords)\
+    \ == 2)\n            if_not_found_x = if_not_found_coords[0]\n            if_not_found_y\
+    \ = if_not_found_coords[1]\n        elif isinstance(if_not_found_coords, dict):\n\
+    \            if_not_found_x = if_not_found_coords[\"x\"]\n            if_not_found_y\
+    \ = if_not_found_coords[\"y\"]\n        else:\n            raise Exception(\"\
+    Expected if_not_found_coords to be None or tuple or list or dict, got %s.\" %\
+    \ (\n                type(if_not_found_coords),))\n\n        keypoints = []\n\
+    \        for i in sm.xrange(nb_keypoints):\n            maxidx_flat = np.argmax(image[...,\
+    \ i])\n            maxidx_ndim = np.unravel_index(maxidx_flat, (height, width))\n\
+    \            found = (image[maxidx_ndim[0], maxidx_ndim[1], i] >= threshold)\n\
+    \            if found:\n                keypoints.append(Keypoint(x=maxidx_ndim[1],\
+    \ y=maxidx_ndim[0]))\n            else:\n                if drop_if_not_found:\n\
+    \                    pass  # dont add the keypoint to the result list, i.e. drop\
+    \ it\n                else:\n                    keypoints.append(Keypoint(x=if_not_found_x,\
+    \ y=if_not_found_y))\n\n        out_shape = (height, width)\n        if nb_channels\
+    \ is not None:\n            out_shape += (nb_channels,)\n        return KeypointsOnImage(keypoints,\
+    \ shape=out_shape)"
+  - "search_document: def choice_action_by_probs(probs=(0.5, 0.5), action_list=None):\n\
+    \    \"\"\"Choice and return an an action by given the action probability distribution.\n\
+    \n    Parameters\n    ------------\n    probs : list of float.\n        The probability\
+    \ distribution of all actions.\n    action_list : None or a list of int or others\n\
+    \        A list of action in integer, string or others. If None, returns an integer\
+    \ range between 0 and len(probs)-1.\n\n    Returns\n    --------\n    float int\
+    \ or str\n        The chosen action.\n\n    Examples\n    ----------\n    >>>\
+    \ for _ in range(5):\n    >>>     a = choice_action_by_probs([0.2, 0.4, 0.4])\n\
+    \    >>>     print(a)\n    0\n    1\n    1\n    2\n    1\n    >>> for _ in range(3):\n\
+    \    >>>     a = choice_action_by_probs([0.5, 0.5], ['a', 'b'])\n    >>>     print(a)\n\
+    \    a\n    b\n    b\n\n    \"\"\"\n    if action_list is None:\n        n_action\
+    \ = len(probs)\n        action_list = np.arange(n_action)\n    else:\n       \
+    \ if len(action_list) != len(probs):\n            raise Exception(\"number of\
+    \ actions should equal to number of probabilities.\")\n    return np.random.choice(action_list,\
+    \ p=probs)"
+  - "search_document: def __validateExperimentControl(self, control):\n    \"\"\"\
+    \ Validates control dictionary for the experiment context\"\"\"\n    # Validate\
+    \ task list\n    taskList = control.get('tasks', None)\n    if taskList is not\
+    \ None:\n      taskLabelsList = []\n\n      for task in taskList:\n        validateOpfJsonValue(task,\
+    \ \"opfTaskSchema.json\")\n        validateOpfJsonValue(task['taskControl'], \"\
+    opfTaskControlSchema.json\")\n\n        taskLabel = task['taskLabel']\n\n    \
+    \    assert isinstance(taskLabel, types.StringTypes), \\\n               \"taskLabel\
+    \ type: %r\" % type(taskLabel)\n        assert len(taskLabel) > 0, \"empty string\
+    \ taskLabel not is allowed\"\n\n        taskLabelsList.append(taskLabel.lower())\n\
+    \n      taskLabelDuplicates = filter(lambda x: taskLabelsList.count(x) > 1,\n\
+    \                                   taskLabelsList)\n      assert len(taskLabelDuplicates)\
+    \ == 0, \\\n             \"Duplcate task labels are not allowed: %s\" % taskLabelDuplicates\n\
+    \n    return"
+- source_sentence: "search_query: Augment endlessly images in the source queue.\n\n\
+    \        This is a worker function for that endlessly queries the source queue\
+    \ (input batches),\n        augments batches in it and sends the result to the\
+    \ output queue."
+  sentences:
+  - "search_document: def _augment_images_worker(self, augseq, queue_source, queue_result,\
+    \ seedval):\n        \"\"\"\n        Augment endlessly images in the source queue.\n\
+    \n        This is a worker function for that endlessly queries the source queue\
+    \ (input batches),\n        augments batches in it and sends the result to the\
+    \ output queue.\n\n        \"\"\"\n        np.random.seed(seedval)\n        random.seed(seedval)\n\
+    \        augseq.reseed(seedval)\n        ia.seed(seedval)\n\n        loader_finished\
+    \ = False\n\n        while not loader_finished:\n            # wait for a new\
+    \ batch in the source queue and load it\n            try:\n                batch_str\
+    \ = queue_source.get(timeout=0.1)\n                batch = pickle.loads(batch_str)\n\
+    \                if batch is None:\n                    loader_finished = True\n\
+    \                    # put it back in so that other workers know that the loading\
+    \ queue is finished\n                    queue_source.put(pickle.dumps(None, protocol=-1))\n\
+    \                else:\n                    batch_aug = augseq.augment_batch(batch)\n\
+    \n                    # send augmented batch to output queue\n               \
+    \     batch_str = pickle.dumps(batch_aug, protocol=-1)\n                    queue_result.put(batch_str)\n\
+    \            except QueueEmpty:\n                time.sleep(0.01)\n\n        queue_result.put(pickle.dumps(None,\
+    \ protocol=-1))\n        time.sleep(0.01)"
+  - "search_document: def show_perf_attrib_stats(returns,\n                      \
+    \     positions,\n                           factor_returns,\n               \
+    \            factor_loadings,\n                           transactions=None,\n\
+    \                           pos_in_dollars=True):\n    \"\"\"\n    Calls `perf_attrib`\
+    \ using inputs, and displays outputs using\n    `utils.print_table`.\n    \"\"\
+    \"\n    risk_exposures, perf_attrib_data = perf_attrib(\n        returns,\n  \
+    \      positions,\n        factor_returns,\n        factor_loadings,\n       \
+    \ transactions,\n        pos_in_dollars=pos_in_dollars,\n    )\n\n    perf_attrib_stats,\
+    \ risk_exposure_stats =\\\n        create_perf_attrib_stats(perf_attrib_data,\
+    \ risk_exposures)\n\n    percentage_formatter = '{:.2%}'.format\n    float_formatter\
+    \ = '{:.2f}'.format\n\n    summary_stats = perf_attrib_stats.loc[['Annualized\
+    \ Specific Return',\n                                           'Annualized Common\
+    \ Return',\n                                           'Annualized Total Return',\n\
+    \                                           'Specific Sharpe Ratio']]\n\n    #\
+    \ Format return rows in summary stats table as percentages.\n    for col_name\
+    \ in (\n        'Annualized Specific Return',\n        'Annualized Common Return',\n\
+    \        'Annualized Total Return',\n    ):\n        summary_stats[col_name] =\
+    \ percentage_formatter(summary_stats[col_name])\n\n    # Display sharpe to two\
+    \ decimal places.\n    summary_stats['Specific Sharpe Ratio'] = float_formatter(\n\
+    \        summary_stats['Specific Sharpe Ratio']\n    )\n\n    print_table(summary_stats,\
+    \ name='Summary Statistics')\n\n    print_table(\n        risk_exposure_stats,\n\
+    \        name='Exposures Summary',\n        # In exposures table, format exposure\
+    \ column to 2 decimal places, and\n        # return columns  as percentages.\n\
+    \        formatters={\n            'Average Risk Factor Exposure': float_formatter,\n\
+    \            'Annualized Return': percentage_formatter,\n            'Cumulative\
+    \ Return': percentage_formatter,\n        },\n    )"
+  - "search_document: def binary_cross_entropy(output, target, epsilon=1e-8, name='bce_loss'):\n\
+    \    \"\"\"Binary cross entropy operation.\n\n    Parameters\n    ----------\n\
+    \    output : Tensor\n        Tensor with type of `float32` or `float64`.\n  \
+    \  target : Tensor\n        The target distribution, format the same with `output`.\n\
+    \    epsilon : float\n        A small value to avoid output to be zero.\n    name\
+    \ : str\n        An optional name to attach to this function.\n\n    References\n\
+    \    -----------\n    - `ericjang-DRAW <https://github.com/ericjang/draw/blob/master/draw.py#L73>`__\n\
+    \n    \"\"\"\n    #     with ops.op_scope([output, target], name, \"bce_loss\"\
+    ) as name:\n    #         output = ops.convert_to_tensor(output, name=\"preds\"\
+    )\n    #         target = ops.convert_to_tensor(targets, name=\"target\")\n\n\
+    \    # with tf.name_scope(name):\n    return tf.reduce_mean(\n        tf.reduce_sum(-(target\
+    \ * tf.log(output + epsilon) + (1. - target) * tf.log(1. - output + epsilon)),\
+    \ axis=1),\n        name=name\n    )"
+- source_sentence: 'search_query: episode_batch: array(batch_size x (T or T+1) x dim_key)'
+  sentences:
+  - "search_document: def get_txn_vol(transactions):\n    \"\"\"\n    Extract daily\
+    \ transaction data from set of transaction objects.\n\n    Parameters\n    ----------\n\
+    \    transactions : pd.DataFrame\n        Time series containing one row per symbol\
+    \ (and potentially\n        duplicate datetime indices) and columns for amount\
+    \ and\n        price.\n\n    Returns\n    -------\n    pd.DataFrame\n        Daily\
+    \ transaction volume and number of shares.\n         - See full explanation in\
+    \ tears.create_full_tear_sheet.\n    \"\"\"\n\n    txn_norm = transactions.copy()\n\
+    \    txn_norm.index = txn_norm.index.normalize()\n    amounts = txn_norm.amount.abs()\n\
+    \    prices = txn_norm.price\n    values = amounts * prices\n    daily_amounts\
+    \ = amounts.groupby(amounts.index).sum()\n    daily_values = values.groupby(values.index).sum()\n\
+    \    daily_amounts.name = \"txn_shares\"\n    daily_values.name = \"txn_volume\"\
+    \n    return pd.concat([daily_values, daily_amounts], axis=1)"
+  - "search_document: def deepcopy(self, exterior=None, label=None):\n        \"\"\
+    \"\n        Create a deep copy of the Polygon object.\n\n        Parameters\n\
+    \        ----------\n        exterior : list of Keypoint or list of tuple or (N,2)\
+    \ ndarray, optional\n            List of points defining the polygon. See `imgaug.Polygon.__init__`\
+    \ for details.\n\n        label : None or str\n            If not None, then the\
+    \ label of the copied object will be set to this value.\n\n        Returns\n \
+    \       -------\n        imgaug.Polygon\n            Deep copy.\n\n        \"\"\
+    \"\n        return Polygon(\n            exterior=np.copy(self.exterior) if exterior\
+    \ is None else exterior,\n            label=self.label if label is None else label\n\
+    \        )"
+  - "search_document: def store_episode(self, episode_batch):\n        \"\"\"episode_batch:\
+    \ array(batch_size x (T or T+1) x dim_key)\n        \"\"\"\n        batch_sizes\
+    \ = [len(episode_batch[key]) for key in episode_batch.keys()]\n        assert\
+    \ np.all(np.array(batch_sizes) == batch_sizes[0])\n        batch_size = batch_sizes[0]\n\
+    \n        with self.lock:\n            idxs = self._get_storage_idx(batch_size)\n\
+    \n            # load inputs into buffers\n            for key in self.buffers.keys():\n\
+    \                self.buffers[key][idxs] = episode_batch[key]\n\n            self.n_transitions_stored\
+    \ += batch_size * self.T"
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: codesearchnet val
+      type: codesearchnet_val
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.8926
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.9453666666666667
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.9545
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.9637666666666667
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.8926
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.31512222222222214
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.19090000000000004
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.09637666666666667
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.8926
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.9453666666666667
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.9545
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.9637666666666667
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.9313201256618757
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.9206047883597835
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.9212040995599341
+      name: Cosine Map@100
+---
+# SentenceTransformer
+This is a [sentence-transformers](https://www.SBERT.net) model trained on the code_search_net dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+## Model Details
+### Model Description
+- **Model Type:** Sentence Transformer
+<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - code_search_net
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+### Model Sources
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+### Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+## Usage
+### Direct Usage (Sentence Transformers)
+First install the Sentence Transformers library:
+```bash
+pip install -U sentence-transformers
+```
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+# Download from the 🤗 Hub
+model = SentenceTransformer("fyaronskiy/english_code_retriever")
+# Run inference
+sentences = [
+    'search_query: episode_batch: array(batch_size x (T or T+1) x dim_key)',
+    'search_document: def store_episode(self, episode_batch):\n        """episode_batch: array(batch_size x (T or T+1) x dim_key)\n        """\n        batch_sizes = [len(episode_batch[key]) for key in episode_batch.keys()]\n        assert np.all(np.array(batch_sizes) == batch_sizes[0])\n        batch_size = batch_sizes[0]\n\n        with self.lock:\n            idxs = self._get_storage_idx(batch_size)\n\n            # load inputs into buffers\n            for key in self.buffers.keys():\n                self.buffers[key][idxs] = episode_batch[key]\n\n            self.n_transitions_stored += batch_size * self.T',
+    'search_document: def get_txn_vol(transactions):\n    """\n    Extract daily transaction data from set of transaction objects.\n\n    Parameters\n    ----------\n    transactions : pd.DataFrame\n        Time series containing one row per symbol (and potentially\n        duplicate datetime indices) and columns for amount and\n        price.\n\n    Returns\n    -------\n    pd.DataFrame\n        Daily transaction volume and number of shares.\n         - See full explanation in tears.create_full_tear_sheet.\n    """\n\n    txn_norm = transactions.copy()\n    txn_norm.index = txn_norm.index.normalize()\n    amounts = txn_norm.amount.abs()\n    prices = txn_norm.price\n    values = amounts * prices\n    daily_amounts = amounts.groupby(amounts.index).sum()\n    daily_values = values.groupby(values.index).sum()\n    daily_amounts.name = "txn_shares"\n    daily_values.name = "txn_volume"\n    return pd.concat([daily_values, daily_amounts], axis=1)',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities)
+# tensor([[1.0000, 0.8384, 0.1236],
+#         [0.8384, 1.0000, 0.1544],
+#         [0.1236, 0.1544, 1.0000]])
+```
+<!--
+### Direct Usage (Transformers)
+<details><summary>Click to see the direct usage in Transformers</summary>
+</details>
+-->
+<!--
+### Downstream Usage (Sentence Transformers)
+You can finetune this model on your own dataset.
+<details><summary>Click to expand</summary>
+</details>
+-->
+<!--
+### Out-of-Scope Use
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+## Evaluation
+### Metrics
+#### Information Retrieval
+* Dataset: `codesearchnet_val`
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.8926     |
+| cosine_accuracy@3   | 0.9454     |
+| cosine_accuracy@5   | 0.9545     |
+| cosine_accuracy@10  | 0.9638     |
+| cosine_precision@1  | 0.8926     |
+| cosine_precision@3  | 0.3151     |
+| cosine_precision@5  | 0.1909     |
+| cosine_precision@10 | 0.0964     |
+| cosine_recall@1     | 0.8926     |
+| cosine_recall@3     | 0.9454     |
+| cosine_recall@5     | 0.9545     |
+| cosine_recall@10    | 0.9638     |
+| **cosine_ndcg@10**  | **0.9313** |
+| cosine_mrr@10       | 0.9206     |
+| cosine_map@100      | 0.9212     |
+<!--
+## Bias, Risks and Limitations
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+<!--
+### Recommendations
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+## Training Details
+### Training Dataset
+#### code_search_net
+* Dataset: code_search_net
+* Size: 1,880,853 training samples
+* Columns: <code>func_documentation_string</code> and <code>func_code_string</code>
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "gather_across_devices": false
+  }
+  ```
+### Evaluation Dataset
+#### code_search_net
+* Dataset: code_search_net
+* Size: 30,000 evaluation samples
+* Columns: <code>func_documentation_string</code> and <code>func_code_string</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | func_documentation_string                                                            | func_code_string                                                                      |
+  |:--------|:-------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
+  | type    | string                                                                               | string                                                                                |
+  | details | <ul><li>min: 9 tokens</li><li>mean: 172.22 tokens</li><li>max: 2122 tokens</li></ul> | <ul><li>min: 52 tokens</li><li>mean: 481.97 tokens</li><li>max: 8192 tokens</li></ul> |
+* Samples:
+  | func_documentation_string                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | func_code_string                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>search_query: Train a deepq model.<br><br>    Parameters<br>    -------<br>    env: gym.Env<br>        environment to train on<br>    network: string or a function<br>        neural network to use as a q function approximator. If string, has to be one of the names of registered models in baselines.common.models<br>        (mlp, cnn, conv_only). If a function, should take an observation tensor and return a latent variable tensor, which<br>        will be mapped to the Q function heads (see build_q_func in baselines.deepq.models for details on that)<br>    seed: int or None<br>        prng seed. The runs with the same seed "should" give the same results. If None, no seeding is used.<br>    lr: float<br>        learning rate for adam optimizer<br>    total_timesteps: int<br>        number of env steps to optimizer for<br>    buffer_size: int<br>        size of the replay buffer<br>    exploration_fraction: float<br>        fraction of entire training period over which the exploration rate is annealed<br>    exploration_final_eps: float<br>        fin...</code> | <code>search_document: def learn(env,<br>          network,<br>          seed=None,<br>          lr=5e-4,<br>          total_timesteps=100000,<br>          buffer_size=50000,<br>          exploration_fraction=0.1,<br>          exploration_final_eps=0.02,<br>          train_freq=1,<br>          batch_size=32,<br>          print_freq=100,<br>          checkpoint_freq=10000,<br>          checkpoint_path=None,<br>          learning_starts=1000,<br>          gamma=1.0,<br>          target_network_update_freq=500,<br>          prioritized_replay=False,<br>          prioritized_replay_alpha=0.6,<br>          prioritized_replay_beta0=0.4,<br>          prioritized_replay_beta_iters=None,<br>          prioritized_replay_eps=1e-6,<br>          param_noise=False,<br>          callback=None,<br>          load_path=None,<br>          **network_kwargs<br>            ):<br>    """Train a deepq model.<br><br>    Parameters<br>    -------<br>    env: gym.Env<br>        environment to train on<br>    network: string or a function<br>        neural network to use as a q function approximator. If string, has ...</code> |
+  | <code>search_query: Save model to a pickle located at `path`</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | <code>search_document: def save_act(self, path=None):<br>        """Save model to a pickle located at `path`"""<br>        if path is None:<br>            path = os.path.join(logger.get_dir(), "model.pkl")<br><br>        with tempfile.TemporaryDirectory() as td:<br>            save_variables(os.path.join(td, "model"))<br>            arc_name = os.path.join(td, "packed.zip")<br>            with zipfile.ZipFile(arc_name, 'w') as zipf:<br>                for root, dirs, files in os.walk(td):<br>                    for fname in files:<br>                        file_path = os.path.join(root, fname)<br>                        if file_path != arc_name:<br>                            zipf.write(file_path, os.path.relpath(file_path, td))<br>            with open(arc_name, "rb") as f:<br>                model_data = f.read()<br>        with open(path, "wb") as f:<br>            cloudpickle.dump((model_data, self._act_params), f)</code>                                                                                                                                                                                |
+  | <code>search_query: CNN from Nature paper.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | <code>search_document: def nature_cnn(unscaled_images, **conv_kwargs):<br>    """<br>    CNN from Nature paper.<br>    """<br>    scaled_images = tf.cast(unscaled_images, tf.float32) / 255.<br>    activ = tf.nn.relu<br>    h = activ(conv(scaled_images, 'c1', nf=32, rf=8, stride=4, init_scale=np.sqrt(2),<br>                   **conv_kwargs))<br>    h2 = activ(conv(h, 'c2', nf=64, rf=4, stride=2, init_scale=np.sqrt(2), **conv_kwargs))<br>    h3 = activ(conv(h2, 'c3', nf=64, rf=3, stride=1, init_scale=np.sqrt(2), **conv_kwargs))<br>    h3 = conv_to_fc(h3)<br>    return activ(fc(h3, 'fc1', nh=512, init_scale=np.sqrt(2)))</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "gather_across_devices": false
+  }
+  ```
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 2
+- `gradient_accumulation_steps`: 32
+- `learning_rate`: 2e-05
+- `max_steps`: 58776
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `resume_from_checkpoint`: ../models/ModernBERT-base_codesearchnet_bs64_lr_2e-05_2nd_epoch/checkpoint-400
+- `batch_sampler`: no_duplicates
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 2
+- `per_device_eval_batch_size`: 8
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 32
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 3.0
+- `max_steps`: 58776
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: ../models/ModernBERT-base_codesearchnet_bs64_lr_2e-05_2nd_epoch/checkpoint-400
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`:
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+</details>
+### Training Logs
+<details><summary>Click to expand</summary>
+| Epoch  | Step  | Training Loss | Validation Loss | codesearchnet_val_cosine_ndcg@10 |
+|:------:|:-----:|:-------------:|:---------------:|:--------------------------------:|
+| 0.0017 | 100   | 0.19          | -               | -                                |
+| 0.0034 | 200   | 0.1583        | -               | -                                |
+| 0.0051 | 300   | 0.6792        | -               | -                                |
+| 0.0068 | 400   | 0.2464        | -               | -                                |
+| 0.0085 | 500   | 0.2829        | 0.1672          | 0.8183                           |
+| 0.0102 | 600   | 0.3795        | -               | -                                |
+| 0.0119 | 700   | 0.2691        | -               | -                                |
+| 0.0136 | 800   | 0.2768        | -               | -                                |
+| 0.0153 | 900   | 0.2254        | -               | -                                |
+| 0.0170 | 1000  | 0.2422        | 0.1815          | 0.8134                           |
+| 0.0187 | 1100  | 0.3755        | -               | -                                |
+| 0.0204 | 1200  | 0.2652        | -               | -                                |
+| 0.0221 | 1300  | 0.2339        | -               | -                                |
+| 0.0238 | 1400  | 0.3393        | -               | -                                |
+| 0.0255 | 1500  | 0.7895        | 0.1694          | 0.8191                           |
+| 0.0272 | 1600  | 0.2055        | -               | -                                |
+| 0.0289 | 1700  | 0.1205        | -               | -                                |
+| 0.0306 | 1800  | 0.198         | -               | -                                |
+| 0.0323 | 1900  | 0.3327        | -               | -                                |
+| 0.0340 | 2000  | 0.1377        | 0.1757          | 0.8136                           |
+| 0.0357 | 2100  | 0.3203        | -               | -                                |
+| 0.0374 | 2200  | 0.3851        | -               | -                                |
+| 0.0391 | 2300  | 0.3489        | -               | -                                |
+| 0.0408 | 2400  | 0.2502        | -               | -                                |
+| 0.0425 | 2500  | 0.1615        | 0.1659          | 0.8216                           |
+| 0.0442 | 2600  | 0.2771        | -               | -                                |
+| 0.0459 | 2700  | 0.2109        | -               | -                                |
+| 0.0476 | 2800  | 0.201         | -               | -                                |
+| 0.0493 | 2900  | 0.1754        | -               | -                                |
+| 0.0510 | 3000  | 0.2428        | 0.1787          | 0.8147                           |
+| 0.0527 | 3100  | 0.1937        | -               | -                                |
+| 0.0544 | 3200  | 0.3725        | -               | -                                |
+| 0.0561 | 3300  | 0.1994        | -               | -                                |
+| 0.0578 | 3400  | 0.2107        | -               | -                                |
+| 0.0595 | 3500  | 0.2011        | 0.1701          | 0.8193                           |
+| 0.0612 | 3600  | 0.3347        | -               | -                                |
+| 0.0630 | 3700  | 5.1378        | -               | -                                |
+| 0.0647 | 3800  | 0.2538        | -               | -                                |
+| 0.0664 | 3900  | 0.248         | -               | -                                |
+| 0.0681 | 4000  | 0.4243        | 0.1866          | 0.8112                           |
+| 0.0698 | 4100  | 0.206         | -               | -                                |
+| 0.0715 | 4200  | 0.3278        | -               | -                                |
+| 0.0732 | 4300  | 0.0888        | -               | -                                |
+| 0.0749 | 4400  | 0.3515        | -               | -                                |
+| 0.0766 | 4500  | 0.6627        | 0.1696          | 0.8165                           |
+| 0.0783 | 4600  | 0.2656        | -               | -                                |
+| 0.0800 | 4700  | 0.1814        | -               | -                                |
+| 0.0817 | 4800  | 0.1112        | -               | -                                |
+| 0.0834 | 4900  | 0.3206        | -               | -                                |
+| 0.0851 | 5000  | 0.172         | 0.2241          | 0.8027                           |
+| 0.0868 | 5100  | 1.1987        | -               | -                                |
+| 0.0885 | 5200  | 0.4428        | -               | -                                |
+| 0.0902 | 5300  | 0.3343        | -               | -                                |
+| 0.0919 | 5400  | 0.7683        | -               | -                                |
+| 0.0936 | 5500  | 0.1866        | 0.1827          | 0.8101                           |
+| 0.0953 | 5600  | 0.3964        | -               | -                                |
+| 0.0970 | 5700  | 0.213         | -               | -                                |
+| 0.0987 | 5800  | 0.2434        | -               | -                                |
+| 0.1004 | 5900  | 0.3023        | -               | -                                |
+| 0.1021 | 6000  | 0.2485        | 0.2432          | 0.7912                           |
+| 0.1038 | 6100  | 0.3683        | -               | -                                |
+| 0.1055 | 6200  | 0.2611        | -               | -                                |
+| 0.1072 | 6300  | 0.3496        | -               | -                                |
+| 0.1089 | 6400  | 0.1196        | -               | -                                |
+| 0.1106 | 6500  | 4.8561        | 0.0884          | 0.8712                           |
+| 0.1123 | 6600  | 5.5351        | -               | -                                |
+| 0.1140 | 6700  | 5.6138        | -               | -                                |
+| 0.1157 | 6800  | 7.7965        | -               | -                                |
+| 0.1174 | 6900  | 4.4967        | -               | -                                |
+| 0.1191 | 7000  | 4.662         | 0.0445          | 0.9163                           |
+| 0.1208 | 7100  | 7.5013        | -               | -                                |
+| 0.1225 | 7200  | 4.7495        | -               | -                                |
+| 0.1242 | 7300  | 3.9834        | -               | -                                |
+| 0.1259 | 7400  | 3.9619        | -               | -                                |
+| 0.1276 | 7500  | 4.4785        | 0.0389          | 0.9195                           |
+| 0.1293 | 7600  | 3.6073        | -               | -                                |
+| 0.1310 | 7700  | 4.5819        | -               | -                                |
+| 0.1327 | 7800  | 3.7317        | -               | -                                |
+| 0.1344 | 7900  | 4.0748        | -               | -                                |
+| 0.1361 | 8000  | 4.1834        | 0.0391          | 0.9166                           |
+| 0.1378 | 8100  | 5.0615        | -               | -                                |
+| 0.1395 | 8200  | 3.3821        | -               | -                                |
+| 0.1412 | 8300  | 3.3912        | -               | -                                |
+| 0.1429 | 8400  | 2.7322        | -               | -                                |
+| 0.1446 | 8500  | 4.5988        | 0.0440          | 0.9142                           |
+| 0.1463 | 8600  | 3.9864        | -               | -                                |
+| 0.1480 | 8700  | 3.9303        | -               | -                                |
+| 0.1497 | 8800  | 4.4749        | -               | -                                |
+| 0.1514 | 8900  | 4.498         | -               | -                                |
+| 0.1531 | 9000  | 7.6013        | 0.0420          | 0.9192                           |
+| 0.1548 | 9100  | 5.5126        | -               | -                                |
+| 0.1565 | 9200  | 6.8746        | -               | -                                |
+| 0.1582 | 9300  | 4.7012        | -               | -                                |
+| 0.1599 | 9400  | 11.9009       | -               | -                                |
+| 0.1616 | 9500  | 3.7235        | 0.0397          | 0.9203                           |
+| 0.1633 | 9600  | 3.5386        | -               | -                                |
+| 0.1650 | 9700  | 5.9226        | -               | -                                |
+| 0.1667 | 9800  | 3.7856        | -               | -                                |
+| 0.1684 | 9900  | 3.3258        | -               | -                                |
+| 0.1701 | 10000 | 4.98          | 0.0448          | 0.9215                           |
+| 0.1718 | 10100 | 4.2472        | -               | -                                |
+| 0.1735 | 10200 | 4.4016        | -               | -                                |
+| 0.1752 | 10300 | 4.8707        | -               | -                                |
+| 0.1769 | 10400 | 4.9904        | -               | -                                |
+| 0.1786 | 10500 | 3.9357        | 0.0383          | 0.9241                           |
+| 0.1803 | 10600 | 4.0991        | -               | -                                |
+| 0.1820 | 10700 | 4.4683        | -               | -                                |
+| 0.1837 | 10800 | 3.4352        | -               | -                                |
+| 0.1854 | 10900 | 3.609         | -               | -                                |
+| 0.1872 | 11000 | 4.4227        | 0.0401          | 0.9242                           |
+| 0.1889 | 11100 | 1.1888        | -               | -                                |
+| 0.1906 | 11200 | 3.2571        | -               | -                                |
+| 0.1923 | 11300 | 3.8446        | -               | -                                |
+| 0.1940 | 11400 | 4.3794        | -               | -                                |
+| 0.1957 | 11500 | 4.1946        | 0.0383          | 0.9238                           |
+| 0.1974 | 11600 | 3.3693        | -               | -                                |
+| 0.1991 | 11700 | 2.5459        | -               | -                                |
+| 0.2008 | 11800 | 4.2474        | -               | -                                |
+| 0.2025 | 11900 | 3.0217        | -               | -                                |
+| 0.2042 | 12000 | 5.5281        | 0.0413          | 0.9196                           |
+| 0.2059 | 12100 | 4.8121        | -               | -                                |
+| 0.2076 | 12200 | 4.6823        | -               | -                                |
+| 0.2093 | 12300 | 3.6668        | -               | -                                |
+| 0.2110 | 12400 | 7.6779        | -               | -                                |
+| 0.2127 | 12500 | 10.2691       | 0.0400          | 0.9188                           |
+| 0.2144 | 12600 | 5.3269        | -               | -                                |
+| 0.2161 | 12700 | 1.9738        | -               | -                                |
+| 0.2178 | 12800 | 0.8293        | -               | -                                |
+| 0.2195 | 12900 | 3.1235        | -               | -                                |
+| 0.2212 | 13000 | 4.3066        | 0.0394          | 0.9205                           |
+| 0.2229 | 13100 | 3.3231        | -               | -                                |
+| 0.2246 | 13200 | 4.0379        | -               | -                                |
+| 0.2263 | 13300 | 2.972         | -               | -                                |
+| 0.2280 | 13400 | 6.0747        | -               | -                                |
+| 0.2297 | 13500 | 4.528         | 0.0388          | 0.9171                           |
+| 0.2314 | 13600 | 4.0589        | -               | -                                |
+| 0.2331 | 13700 | 3.5025        | -               | -                                |
+| 0.2348 | 13800 | 4.2629        | -               | -                                |
+| 0.2365 | 13900 | 4.6222        | -               | -                                |
+| 0.2382 | 14000 | 5.2728        | 0.0369          | 0.9164                           |
+| 0.2399 | 14100 | 4.7144        | -               | -                                |
+| 0.2416 | 14200 | 6.1632        | -               | -                                |
+| 0.2433 | 14300 | 4.692         | -               | -                                |
+| 0.2450 | 14400 | 4.7192        | -               | -                                |
+| 0.2467 | 14500 | 4.4553        | 0.0383          | 0.9151                           |
+| 0.2484 | 14600 | 5.5914        | -               | -                                |
+| 0.2501 | 14700 | 5.1508        | -               | -                                |
+| 0.2518 | 14800 | 5.6841        | -               | -                                |
+| 0.2535 | 14900 | 5.0169        | -               | -                                |
+| 0.2552 | 15000 | 4.9999        | 0.0411          | 0.9054                           |
+| 0.2569 | 15100 | 4.169         | -               | -                                |
+| 0.2586 | 15200 | 4.2057        | -               | -                                |
+| 0.2603 | 15300 | 4.2864        | -               | -                                |
+| 0.2620 | 15400 | 4.7846        | -               | -                                |
+| 0.2637 | 15500 | 3.7807        | 0.0414          | 0.9063                           |
+| 0.2654 | 15600 | 1.1362        | -               | -                                |
+| 0.2671 | 15700 | 2.2033        | -               | -                                |
+| 0.2688 | 15800 | 1.1895        | -               | -                                |
+| 0.2705 | 15900 | 3.3078        | -               | -                                |
+| 0.2722 | 16000 | 0.8944        | 0.0480          | 0.8973                           |
+| 0.2739 | 16100 | 0.1877        | -               | -                                |
+| 0.2756 | 16200 | 0.6408        | -               | -                                |
+| 0.2773 | 16300 | 1.0533        | -               | -                                |
+| 0.2790 | 16400 | 0.5994        | -               | -                                |
+| 0.2807 | 16500 | 0.4108        | 0.0594          | 0.8605                           |
+| 0.2824 | 16600 | 0.2404        | -               | -                                |
+| 0.2841 | 16700 | 0.0997        | -               | -                                |
+| 0.2858 | 16800 | 0.1914        | -               | -                                |
+| 0.2875 | 16900 | 0.2929        | -               | -                                |
+| 0.2892 | 17000 | 1.6005        | 0.0527          | 0.8789                           |
+| 0.2909 | 17100 | 1.222         | -               | -                                |
+| 0.2926 | 17200 | 0.9074        | -               | -                                |
+| 0.2943 | 17300 | 1.035         | -               | -                                |
+| 0.2960 | 17400 | 0.9031        | -               | -                                |
+| 0.2977 | 17500 | 0.6543        | 0.0532          | 0.8673                           |
+| 0.2994 | 17600 | 0.6473        | -               | -                                |
+| 0.3011 | 17700 | 0.8602        | -               | -                                |
+| 0.3028 | 17800 | 1.1188        | -               | -                                |
+| 0.3045 | 17900 | 1.0075        | -               | -                                |
+| 0.3062 | 18000 | 1.882         | 0.0517          | 0.8728                           |
+| 0.3079 | 18100 | 2.7281        | -               | -                                |
+| 0.3097 | 18200 | 1.3526        | -               | -                                |
+| 0.3114 | 18300 | 0.8499        | -               | -                                |
+| 0.3131 | 18400 | 1.4446        | -               | -                                |
+| 0.3148 | 18500 | 0.868         | 0.0576          | 0.8497                           |
+| 0.3165 | 18600 | 0.24          | -               | -                                |
+| 0.3182 | 18700 | 0.1584        | -               | -                                |
+| 0.3199 | 18800 | 0.6843        | -               | -                                |
+| 0.3216 | 18900 | 0.6952        | -               | -                                |
+| 0.3233 | 19000 | 0.9405        | 0.0540          | 0.8720                           |
+| 0.3250 | 19100 | 1.0507        | -               | -                                |
+| 0.3267 | 19200 | 1.3594        | -               | -                                |
+| 0.3284 | 19300 | 1.088         | -               | -                                |
+| 0.3301 | 19400 | 5.7851        | -               | -                                |
+| 0.3318 | 19500 | 1.389         | 0.0481          | 0.8735                           |
+| 0.3335 | 19600 | 0.8319        | -               | -                                |
+| 0.3352 | 19700 | 1.3223        | -               | -                                |
+| 0.3369 | 19800 | 1.483         | -               | -                                |
+| 0.3386 | 19900 | 1.4582        | -               | -                                |
+| 0.3403 | 20000 | 1.3791        | 0.0523          | 0.8568                           |
+| 0.3420 | 20100 | 0.9329        | -               | -                                |
+| 0.3437 | 20200 | 1.4642        | -               | -                                |
+| 0.3454 | 20300 | 1.0606        | -               | -                                |
+| 0.3471 | 20400 | 1.36          | -               | -                                |
+| 0.3488 | 20500 | 2.2963        | 0.0491          | 0.8852                           |
+| 0.3505 | 20600 | 2.1325        | -               | -                                |
+| 0.3522 | 20700 | 2.7292        | -               | -                                |
+| 0.3539 | 20800 | 3.318         | -               | -                                |
+| 0.3556 | 20900 | 2.9986        | -               | -                                |
+| 0.3573 | 21000 | 2.795         | 0.0404          | 0.9025                           |
+| 0.3590 | 21100 | 3.2957        | -               | -                                |
+| 0.3607 | 21200 | 3.332         | -               | -                                |
+| 0.3624 | 21300 | 3.6357        | -               | -                                |
+| 0.3641 | 21400 | 3.2703        | -               | -                                |
+| 0.3658 | 21500 | 3.1093        | 0.0409          | 0.9038                           |
+| 0.3675 | 21600 | 3.0346        | -               | -                                |
+| 0.3692 | 21700 | 2.9721        | -               | -                                |
+| 0.3709 | 21800 | 2.8401        | -               | -                                |
+| 0.3726 | 21900 | 3.6846        | -               | -                                |
+| 0.3743 | 22000 | 3.7735        | 0.0449          | 0.9011                           |
+| 0.3760 | 22100 | 4.5612        | -               | -                                |
+| 0.3777 | 22200 | 2.9749        | -               | -                                |
+| 0.3794 | 22300 | 4.1631        | -               | -                                |
+| 0.3811 | 22400 | 3.5604        | -               | -                                |
+| 0.3828 | 22500 | 3.1771        | 0.0465          | 0.8791                           |
+| 0.3845 | 22600 | 4.0018        | -               | -                                |
+| 0.3862 | 22700 | 4.5043        | -               | -                                |
+| 0.3879 | 22800 | 3.9448        | -               | -                                |
+| 0.3896 | 22900 | 5.3978        | -               | -                                |
+| 0.3913 | 23000 | 1.7375        | 0.0423          | 0.8897                           |
+| 0.3930 | 23100 | 1.3611        | -               | -                                |
+| 0.3947 | 23200 | 1.2127        | -               | -                                |
+| 0.3964 | 23300 | 2.5636        | -               | -                                |
+| 0.3981 | 23400 | 3.5466        | -               | -                                |
+| 0.3998 | 23500 | 2.8656        | 0.0424          | 0.8979                           |
+| 0.4015 | 23600 | 2.7182        | -               | -                                |
+| 0.4032 | 23700 | 2.8646        | -               | -                                |
+| 0.4049 | 23800 | 2.9465        | -               | -                                |
+| 0.4066 | 23900 | 2.9163        | -               | -                                |
+| 0.4083 | 24000 | 2.9331        | 0.0463          | 0.8938                           |
+| 0.4100 | 24100 | 3.9935        | -               | -                                |
+| 0.4117 | 24200 | 2.7475        | -               | -                                |
+| 0.4134 | 24300 | 5.1587        | -               | -                                |
+| 0.4151 | 24400 | 3.7547        | -               | -                                |
+| 0.4168 | 24500 | 2.0947        | 0.0474          | 0.8835                           |
+| 0.4185 | 24600 | 2.4338        | -               | -                                |
+| 0.4202 | 24700 | 3.7207        | -               | -                                |
+| 0.4219 | 24800 | 3.4291        | -               | -                                |
+| 0.4236 | 24900 | 3.3418        | -               | -                                |
+| 0.4253 | 25000 | 2.5511        | 0.0430          | 0.9044                           |
+| 0.4270 | 25100 | 2.983         | -               | -                                |
+| 0.4287 | 25200 | 2.3925        | -               | -                                |
+| 0.4304 | 25300 | 3.0416        | -               | -                                |
+| 0.4321 | 25400 | 2.7997        | -               | -                                |
+| 0.4339 | 25500 | 2.7176        | 0.0444          | 0.8976                           |
+| 0.4356 | 25600 | 2.9507        | -               | -                                |
+| 0.4373 | 25700 | 2.3705        | -               | -                                |
+| 0.4390 | 25800 | 4.2975        | -               | -                                |
+| 0.4407 | 25900 | 3.527         | -               | -                                |
+| 0.4424 | 26000 | 2.3073        | 0.0453          | 0.8868                           |
+| 0.4441 | 26100 | 3.0645        | -               | -                                |
+| 0.4458 | 26200 | 3.4071        | -               | -                                |
+| 0.4475 | 26300 | 3.7307        | -               | -                                |
+| 0.4492 | 26400 | 2.6497        | -               | -                                |
+| 0.4509 | 26500 | 2.668         | 0.0483          | 0.8826                           |
+| 0.4526 | 26600 | 2.8279        | -               | -                                |
+| 0.4543 | 26700 | 3.6331        | -               | -                                |
+| 0.4560 | 26800 | 2.8807        | -               | -                                |
+| 0.4577 | 26900 | 3.5984        | -               | -                                |
+| 0.4594 | 27000 | 2.9723        | 0.0442          | 0.8921                           |
+| 0.4611 | 27100 | 2.5831        | -               | -                                |
+| 0.4628 | 27200 | 2.9105        | -               | -                                |
+| 0.4645 | 27300 | 3.3101        | -               | -                                |
+| 0.4662 | 27400 | 3.0373        | -               | -                                |
+| 0.4679 | 27500 | 2.8517        | 0.0449          | 0.9052                           |
+| 0.4696 | 27600 | 2.8638        | -               | -                                |
+| 0.4713 | 27700 | 3.1508        | -               | -                                |
+| 0.4730 | 27800 | 2.3221        | -               | -                                |
+| 0.4747 | 27900 | 2.589         | -               | -                                |
+| 0.4764 | 28000 | 3.006         | 0.0481          | 0.8875                           |
+| 0.4781 | 28100 | 2.7114        | -               | -                                |
+| 0.4798 | 28200 | 2.9843        | -               | -                                |
+| 0.4815 | 28300 | 3.2839        | -               | -                                |
+| 0.4832 | 28400 | 3.3347        | -               | -                                |
+| 0.4849 | 28500 | 3.2885        | 0.0463          | 0.8951                           |
+| 0.4866 | 28600 | 3.1835        | -               | -                                |
+| 0.4883 | 28700 | 2.6282        | -               | -                                |
+| 0.4900 | 28800 | 2.2738        | -               | -                                |
+| 0.4917 | 28900 | 3.1998        | -               | -                                |
+| 0.4934 | 29000 | 2.9384        | 0.0451          | 0.8981                           |
+| 0.4951 | 29100 | 4.6047        | -               | -                                |
+| 0.4968 | 29200 | 3.0966        | -               | -                                |
+| 0.4985 | 29300 | 3.1956        | -               | -                                |
+| 1.0002 | 29400 | 3.4994        | -               | -                                |
+| 1.0019 | 29500 | 0.2776        | 0.0461          | 0.9105                           |
+| 1.0036 | 29600 | 0.2346        | -               | -                                |
+| 1.0053 | 29700 | 0.6977        | -               | -                                |
+| 1.0070 | 29800 | 0.2521        | -               | -                                |
+| 1.0087 | 29900 | 0.3457        | -               | -                                |
+| 1.0104 | 30000 | 0.33          | 0.1606          | 0.8305                           |
+| 1.0121 | 30100 | 0.2394        | -               | -                                |
+| 1.0138 | 30200 | 0.2567        | -               | -                                |
+| 1.0155 | 30300 | 0.2705        | -               | -                                |
+| 1.0172 | 30400 | 0.2423        | -               | -                                |
+| 1.0189 | 30500 | 0.3367        | 0.1255          | 0.8458                           |
+| 1.0206 | 30600 | 0.3427        | -               | -                                |
+| 1.0223 | 30700 | 0.288         | -               | -                                |
+| 1.0240 | 30800 | 0.2142        | -               | -                                |
+| 1.0257 | 30900 | 0.7273        | -               | -                                |
+| 1.0274 | 31000 | 0.1669        | 0.1504          | 0.8340                           |
+| 1.0291 | 31100 | 0.136         | -               | -                                |
+| 1.0308 | 31200 | 0.1797        | -               | -                                |
+| 1.0325 | 31300 | 0.3307        | -               | -                                |
+| 1.0342 | 31400 | 0.1718        | -               | -                                |
+| 1.0359 | 31500 | 0.2782        | 0.1558          | 0.8309                           |
+| 1.0376 | 31600 | 0.3898        | -               | -                                |
+| 1.0393 | 31700 | 0.3348        | -               | -                                |
+| 1.0410 | 31800 | 0.2429        | -               | -                                |
+| 1.0427 | 31900 | 0.1551        | -               | -                                |
+| 1.0444 | 32000 | 0.2651        | 0.1941          | 0.8126                           |
+| 1.0461 | 32100 | 0.2108        | -               | -                                |
+| 1.0478 | 32200 | 0.1574        | -               | -                                |
+| 1.0495 | 32300 | 0.2025        | -               | -                                |
+| 1.0512 | 32400 | 0.211         | -               | -                                |
+| 1.0529 | 32500 | 0.2047        | 0.1675          | 0.8239                           |
+| 1.0546 | 32600 | 0.3256        | -               | -                                |
+| 1.0563 | 32700 | 0.1979        | -               | -                                |
+| 1.0581 | 32800 | 0.2014        | -               | -                                |
+| 1.0598 | 32900 | 0.1971        | -               | -                                |
+| 1.0615 | 33000 | 2.5198        | 0.1667          | 0.8226                           |
+| 1.0632 | 33100 | 2.9456        | -               | -                                |
+| 1.0649 | 33200 | 0.227         | -               | -                                |
+| 1.0666 | 33300 | 0.2095        | -               | -                                |
+| 1.0683 | 33400 | 0.3852        | -               | -                                |
+| 1.0700 | 33500 | 0.189         | 0.1985          | 0.8078                           |
+| 1.0717 | 33600 | 0.3144        | -               | -                                |
+| 1.0734 | 33700 | 0.1082        | -               | -                                |
+| 1.0751 | 33800 | 0.3401        | -               | -                                |
+| 1.0768 | 33900 | 0.6729        | -               | -                                |
+| 1.0785 | 34000 | 0.2654        | 0.1725          | 0.8184                           |
+| 1.0802 | 34100 | 0.1091        | -               | -                                |
+| 1.0819 | 34200 | 0.125         | -               | -                                |
+| 1.0836 | 34300 | 0.254         | -               | -                                |
+| 1.0853 | 34400 | 0.2993        | -               | -                                |
+| 1.0870 | 34500 | 1.0362        | 0.1959          | 0.8129                           |
+| 1.0887 | 34600 | 0.3466        | -               | -                                |
+| 1.0904 | 34700 | 0.4128        | -               | -                                |
+| 1.0921 | 34800 | 0.6896        | -               | -                                |
+| 1.0938 | 34900 | 0.1564        | -               | -                                |
+| 1.0955 | 35000 | 0.3684        | 0.1789          | 0.8180                           |
+| 1.0972 | 35100 | 0.175         | -               | -                                |
+| 1.0989 | 35200 | 0.2262        | -               | -                                |
+| 1.1006 | 35300 | 0.3144        | -               | -                                |
+| 1.1023 | 35400 | 0.1309        | -               | -                                |
+| 1.1040 | 35500 | 0.3343        | 0.1689          | 0.8211                           |
+| 1.1057 | 35600 | 0.2455        | -               | -                                |
+| 1.1074 | 35700 | 0.2755        | -               | -                                |
+| 1.1091 | 35800 | 0.1197        | -               | -                                |
+| 1.1108 | 35900 | 4.0627        | -               | -                                |
+| 1.1125 | 36000 | 5.2316        | 0.0465          | 0.9154                           |
+| 1.1142 | 36100 | 4.9503        | -               | -                                |
+| 1.1159 | 36200 | 5.5472        | -               | -                                |
+| 1.1176 | 36300 | 2.4129        | -               | -                                |
+| 1.1193 | 36400 | 3.0761        | -               | -                                |
+| 1.1210 | 36500 | 5.5435        | 0.0355          | 0.9312                           |
+| 1.1227 | 36600 | 3.123         | -               | -                                |
+| 1.1244 | 36700 | 2.6283        | -               | -                                |
+| 1.1261 | 36800 | 2.9838        | -               | -                                |
+| 1.1278 | 36900 | 2.5708        | -               | -                                |
+| 1.1295 | 37000 | 2.8255        | 0.0347          | 0.9299                           |
+| 1.1312 | 37100 | 2.9355        | -               | -                                |
+| 1.1329 | 37200 | 2.3452        | -               | -                                |
+| 1.1346 | 37300 | 2.9868        | -               | -                                |
+| 1.1363 | 37400 | 2.4357        | -               | -                                |
+| 1.1380 | 37500 | 3.6657        | 0.0342          | 0.9312                           |
+| 1.1397 | 37600 | 2.1038        | -               | -                                |
+| 1.1414 | 37700 | 2.1265        | -               | -                                |
+| 1.1431 | 37800 | 2.5661        | -               | -                                |
+| 1.1448 | 37900 | 3.0195        | -               | -                                |
+| 1.1465 | 38000 | 2.3143        | 0.0356          | 0.9295                           |
+| 1.1482 | 38100 | 2.5888        | -               | -                                |
+| 1.1499 | 38200 | 2.9752        | -               | -                                |
+| 1.1516 | 38300 | 3.7307        | -               | -                                |
+| 1.1533 | 38400 | 5.848         | -               | -                                |
+| 1.1550 | 38500 | 4.4071        | 0.0337          | 0.9300                           |
+| 1.1567 | 38600 | 5.4725        | -               | -                                |
+| 1.1584 | 38700 | 3.4716        | -               | -                                |
+| 1.1601 | 38800 | 11.3059       | -               | -                                |
+| 1.1618 | 38900 | 2.6385        | -               | -                                |
+| 1.1635 | 39000 | 2.218         | 0.0358          | 0.9316                           |
+| 1.1652 | 39100 | 4.2273        | -               | -                                |
+| 1.1669 | 39200 | 2.6294        | -               | -                                |
+| 1.1686 | 39300 | 2.133         | -               | -                                |
+| 1.1703 | 39400 | 3.6478        | -               | -                                |
+| 1.1720 | 39500 | 3.4377        | 0.0378          | 0.9303                           |
+| 1.1737 | 39600 | 2.4562        | -               | -                                |
+| 1.1754 | 39700 | 3.5861        | -               | -                                |
+| 1.1771 | 39800 | 3.0256        | -               | -                                |
+| 1.1788 | 39900 | 3.0825        | -               | -                                |
+| 1.1805 | 40000 | 2.6703        | 0.0322          | 0.9312                           |
+| 1.1823 | 40100 | 3.0079        | -               | -                                |
+| 1.1840 | 40200 | 2.4013        | -               | -                                |
+| 1.1857 | 40300 | 2.6008        | -               | -                                |
+| 1.1874 | 40400 | 2.6009        | -               | -                                |
+| 1.1891 | 40500 | 1.0118        | 0.0364          | 0.9251                           |
+| 1.1908 | 40600 | 1.9537        | -               | -                                |
+| 1.1925 | 40700 | 3.0583        | -               | -                                |
+| 1.1942 | 40800 | 3.0605        | -               | -                                |
+| 1.1959 | 40900 | 2.899         | -               | -                                |
+| 1.1976 | 41000 | 2.1811        | 0.0372          | 0.9314                           |
+| 1.1993 | 41100 | 1.6352        | -               | -                                |
+| 1.2010 | 41200 | 2.897         | -               | -                                |
+| 1.2027 | 41300 | 1.9675        | -               | -                                |
+| 1.2044 | 41400 | 4.1303        | -               | -                                |
+| 1.2061 | 41500 | 3.5249        | 0.0347          | 0.9296                           |
+| 1.2078 | 41600 | 2.9009        | -               | -                                |
+| 1.2095 | 41700 | 2.1829        | -               | -                                |
+| 1.2112 | 41800 | 8.6371        | -               | -                                |
+| 1.2129 | 41900 | 7.3269        | -               | -                                |
+| 1.2146 | 42000 | 4.7817        | 0.0368          | 0.9294                           |
+| 1.2163 | 42100 | 1.0625        | -               | -                                |
+| 1.2180 | 42200 | 0.4482        | -               | -                                |
+| 1.2197 | 42300 | 2.273         | -               | -                                |
+| 1.2214 | 42400 | 2.9058        | -               | -                                |
+| 1.2231 | 42500 | 2.3138        | 0.0340          | 0.9295                           |
+| 1.2248 | 42600 | 2.8566        | -               | -                                |
+| 1.2265 | 42700 | 2.0118        | -               | -                                |
+| 1.2282 | 42800 | 4.6103        | -               | -                                |
+| 1.2299 | 42900 | 3.2476        | -               | -                                |
+| 1.2316 | 43000 | 2.5581        | 0.0323          | 0.9303                           |
+| 1.2333 | 43100 | 2.2805        | -               | -                                |
+| 1.2350 | 43200 | 2.5965        | -               | -                                |
+| 1.2367 | 43300 | 2.6197        | -               | -                                |
+| 1.2384 | 43400 | 3.7837        | -               | -                                |
+| 1.2401 | 43500 | 3.0483        | 0.0316          | 0.9306                           |
+| 1.2418 | 43600 | 3.6523        | -               | -                                |
+| 1.2435 | 43700 | 2.9768        | -               | -                                |
+| 1.2452 | 43800 | 2.844         | -               | -                                |
+| 1.2469 | 43900 | 3.2135        | -               | -                                |
+| 1.2486 | 44000 | 3.5638        | 0.0325          | 0.9313                           |
+</details>
+### Framework Versions
+- Python: 3.10.11
+- Sentence Transformers: 5.1.0
+- Transformers: 4.52.3
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.10.0
+- Datasets: 3.6.0
+- Tokenizers: 0.21.4
+## Citation
+### BibTeX
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+<!--
+## Glossary
+*Clearly define terms in order to be accessible across audiences.*
+-->
+<!--
+## Model Card Authors
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+<!--
+## Model Card Contact
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json ADDED Viewed

	@@ -0,0 +1,45 @@

+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.56.2",
+  "vocab_size": 50368
+}

config_sentence_transformers.json ADDED Viewed

	@@ -0,0 +1,16 @@

+{
+  "model_type": "SentenceTransformer",
+  "__version__": {
+    "sentence_transformers": "5.1.1",
+    "transformers": "4.56.2",
+    "pytorch": "2.8.0+cu126"
+  },
+  "prompts": {
+    "query": "",
+    "document": "",
+    "search_query": "search_query: ",
+    "search_document": "search_document: "
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b8fedc806ca505df9c90c86c57b65f7add28ae0f9b4af7bdc9a67aede4ed6b73
+size 596070136

modules.json ADDED Viewed

	@@ -0,0 +1,14 @@

+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json ADDED Viewed

	@@ -0,0 +1,4 @@

+{
+    "max_seq_length": 8192,
+    "do_lower_case": false
+}

special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,37 @@

+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,952 @@

+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 8192,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}