macadeliccc/laser-polyglot-4x7b

Polyglot-4x7b-24b

Polyglot-4x7b is a Mixture of Experts approach to a multilingual model.

This project is an experiment to see if each expert can be of a different language. The answer is yes.

The model is a merge of models that are capable of Chinese and Japanese output.

• teknium/OpenHermes-2.5-Mistral-7B
• oshizo/japanese-e5-mistral-7b_slerp
• cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
• s3nh/Mistral-7B-Evol-Instruct-Chinese

TODO:

1. [] polyglot tokenizer

Other polyglot models

• macadeliccc/Polyglot-8x7b-v0.1 (adds 3 more languages)

Code Example

Inference Colab Live demo available on Spaces

from transformers import AutoModelForCausalLM, AutoTokenizer

def generate_response(prompt):
    """
    Generate a response from the model based on the input prompt.

    Args:
    prompt (str): Prompt for the model.

    Returns:
    str: The generated response from the model.
    """
    # Tokenize the input prompt
    inputs = tokenizer(prompt, return_tensors="pt")

    # Generate output tokens
    outputs = model.generate(**inputs, max_new_tokens=256, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id)

    # Decode the generated tokens to a string
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)

    return response

# Load the model and tokenizer
model_id = "macadeliccc/laser-polyglot-4x7b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)

# Example prompts in different languages
english_prompt = "Write a quicksort algorithm in python"
chinese_prompt = "用Python写一个快速排序算法"
japanese_prompt = "Pythonでクイックソートアルゴリズムを書いてください"

# Generate and print responses for each language
print("English Response:")
print(generate_response(english_prompt), "\n")

print("Chinese Response:")
print(generate_response(chinese_prompt), "\n")

print("Japanese Response:")
print(generate_response(japanese_prompt), "\n")

Example Output

English:

Write a quicksort algorithm in python.

def quicksort(arr):
    if len(arr) <= 1:
        return arr
    else:
        pivot = arr[0]
        less = [i for i in arr[1:] if i <= pivot]
        greater = [i for i in arr[1:] if i > pivot]
        return quicksort(less) + [pivot] + quicksort(greater)

arr = [5, 2, 9, 1, 5, 7, 4, 8, 6, 3]
print(quicksort(arr))

This is a simple implementation of the quicksort algorithm in python. The function quicksort takes an array as input and returns a sorted array. The algorithm works by selecting a pivot element from the array and partitioning the other elements into two sub-arrays, according to whether they are less than or greater than the pivot. The process is then repeated recursively on the sub-arrays until the entire array is sorted.

Chinese Response:

用Python写一个快速排序算法

def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    else:
        pivot = arr[0]
        less = [i for i in arr[1:] if i <= pivot]
        greater = [i for i in arr[1:] if i > pivot]
        return quick_sort(less) + [pivot] + quick_sort(greater)

arr = [3, 5, 2, 1, 4, 6, 8, 7]
print(quick_sort(arr))

这个程序的时间复杂度为O(nlogn)，空间复杂度为O(n)。

Japanese Response:

Pythonでクイックソートアルゴリズムを書いてください。

def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[0]
    left = [x for x in arr[1:] if x < pivot]
    right = [x for x in arr[1:] if x >= pivot]
    return quicksort(left) + [pivot] + quicksort(right)

print(quicksort([3,6,8,10,1,5,9,2,4,7]))

このコードはクイックソートアルゴリズムを実装しています。クイックソートは一種の分割と conquers アルゴリズムで、配列を分割し、それぞれの部分配列を再帰的にソートします。

この実装では、配列の最初の要素をピボットとして使用します。そして、配列を2つの

Evaluations

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
arc_challenge	Yaml	none	0	acc	0.5495	±	0.0145
		none	0	acc_norm	0.5794	±	0.0144
arc_easy	Yaml	none	0	acc	0.8304	±	0.0077
		none	0	acc_norm	0.8068	±	0.0081
boolq	Yaml	none	0	acc	0.8749	±	0.0058
hellaswag	Yaml	none	0	acc	0.6276	±	0.0048
		none	0	acc_norm	0.8157	±	0.0039
openbookqa	Yaml	none	0	acc	0.3180	±	0.0208
		none	0	acc_norm	0.4460	±	0.0223
piqa	Yaml	none	0	acc	0.8139	±	0.0091
		none	0	acc_norm	0.8237	±	0.0089
winogrande	Yaml	none	0	acc	0.7419	±	0.0123

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	65.79
AI2 Reasoning Challenge (25-Shot)	64.16
HellaSwag (10-Shot)	84.98
MMLU (5-Shot)	63.88
TruthfulQA (0-shot)	55.47
Winogrande (5-shot)	77.82
GSM8k (5-shot)	48.45