Mistral AI 又又又开源了闭源企业级模型——Mistral-Small-Instruct-2409

就在不久前，Mistral 公司在开源了 Pixtral 12B 视觉多模态大模型之后，又开源了自家的企业级小型模型 Mistral-Small-Instruct-2409 （22B），这是 Mistral AI 最新的企业级小型模型，是 Mistral Small v24.02 的升级版。该机型可根据 Mistral Research License 使用，为客户提供了灵活的选择，使其能够在翻译、摘要、情感分析和其他不需要完整通用模型的任务中，选择经济高效、快速可靠的解决方案。
在这里插入图片描述

Mistral Small 雏形采用 Mixtral-8X7B-v0.1（46.7B），这是一个具有 12B 活动参数的稀疏专家混合模型。它的推理能力更强，功能更多，可以生成和推理代码，并且是多语言的，支持英语、法语、德语、意大利语和西班牙语。

太激动人心了， Mistral 型号的性能总是出类拔萃。现在，我们在很多缝隙上都有了出色的覆盖范围

  • 8b- Llama 3.1 8b

  • 12b- Nemo 12b

  • 22b- Mistral Small

  • 27b- Gemma-2 27b

  • 35b- Command-R 35b 08-2024

  • 40-60b- GAP (我相信这里有两个新的 MOE，但我最后发现 Llamacpp 不支持它们)

  • 70b- Llama 3.1 70b

  • 103b- Command-R+ 103b

  • 123b- Mistral Large 2

  • 141b- WizardLM-2 8x22b

  • 230b- Deepseek V2/2.5

  • 405b- Llama 3.1 405b

Mistral Small v24.09 拥有 220 亿个参数，为客户提供了介于 Mistral NeMo 12B 和 Mistral Large 2 之间的便捷中间点，提供了可在各种平台和环境中部署的经济高效的解决方案。。

在这里插入图片描述
在这里插入图片描述

Mistral Small v24.09 拥有 220 亿个参数，为客户提供了介于 Mistral NeMo 12B 和 Mistral Large 2 之间的便捷中间点，提供了可在各种平台和环境中部署的经济高效的解决方案。如下图所示，与以前的模型相比，新的小型模型在人类对齐、推理能力和代码方面都有显著改进。
在这里插入图片描述
在这里插入图片描述

Mistral-Small-Instruct-2409 是一个指示微调版本，具有以下特点：

  • 22B 参数
  • 词汇量达 32768
  • 支持函数调用
  • 128k 序列长度

使用

vLLM（推荐）

安装 vLLM >= v0.6.1.post1

pip install --upgrade vllm

安装 mistral_common >= 1.4.1

pip install --upgrade mistral_common

本地

from vllm import LLM
from vllm.sampling_params import SamplingParamsmodel_name = "mistralai/Mistral-Small-Instruct-2409"sampling_params = SamplingParams(max_tokens=8192)# note that running Mistral-Small on a single GPU requires at least 44 GB of GPU RAM
# If you want to divide the GPU requirement over multiple devices, please add *e.g.* `tensor_parallel=2`
llm = LLM(model=model_name, tokenizer_mode="mistral", config_format="mistral", load_format="mistral")prompt = "How often does the letter r occur in Mistral?"messages = [{"role": "user","content": prompt},
]outputs = llm.chat(messages, sampling_params=sampling_params)print(outputs[0].outputs[0].text)

服务器

vllm serve mistralai/Mistral-Small-Instruct-2409 --tokenizer_mode mistral --config_format mistral --load_format mistral

注意： 在单 GPU 上运行 Mistral-Small 至少需要 44 GB GPU 内存。

如果要将 GPU 需求分配给多个设备，请添加 --tensor_parallel=2 等信息

客户端

curl --location 'http://<your-node-url>:8000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer token' \
--data '{"model": "mistralai/Mistral-Small-Instruct-2409","messages": [{"role": "user","content": "How often does the letter r occur in Mistral?"}]
}'

Mistral-inference

安装mistral_inference >= 1.4.1

pip install mistral_inference --upgrade

下载

from huggingface_hub import snapshot_download
from pathlib import Pathmistral_models_path = Path.home().joinpath('mistral_models', '22B-Instruct-Small')
mistral_models_path.mkdir(parents=True, exist_ok=True)snapshot_download(repo_id="mistralai/Mistral-Small-Instruct-2409", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)

聊天

mistral-chat $HOME/mistral_models/22B-Instruct-Small --instruct --max_tokens 256

Instruct following

from mistral_inference.transformer import Transformer
from mistral_inference.generate import generatefrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequesttokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3")
model = Transformer.from_folder(mistral_models_path)completion_request = ChatCompletionRequest(messages=[UserMessage(content="How often does the letter r occur in Mistral?")])tokens = tokenizer.encode_chat_completion(completion_request).tokensout_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])print(result)

Function calling

from mistral_common.protocol.instruct.tool_calls import Function, Tool
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generatefrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequesttokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3")
model = Transformer.from_folder(mistral_models_path)completion_request = ChatCompletionRequest(tools=[Tool(function=Function(name="get_current_weather",description="Get the current weather",parameters={"type": "object","properties": {"location": {"type": "string","description": "The city and state, e.g. San Francisco, CA",},"format": {"type": "string","enum": ["celsius", "fahrenheit"],"description": "The temperature unit to use. Infer this from the users location.",},},"required": ["location", "format"],},))],messages=[UserMessage(content="What's the weather like today in Paris?"),],
)tokens = tokenizer.encode_chat_completion(completion_request).tokensout_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])print(result)

Hugging Face Transformers

from transformers import LlamaTokenizerFast, MistralForCausalLM
import torchdevice = "cuda"
tokenizer = LlamaTokenizerFast.from_pretrained('mistralai/Mistral-Small-Instruct-2409')
tokenizer.pad_token = tokenizer.eos_tokenmodel = MistralForCausalLM.from_pretrained('mistralai/Mistral-Small-Instruct-2409', torch_dtype=torch.bfloat16)
model = model.to(device)prompt = "How often does the letter r occur in Mistral?"messages = [{"role": "user", "content": prompt},]model_input = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(device)
gen = model.generate(model_input, max_new_tokens=150)
dec = tokenizer.batch_decode(gen)
print(dec)

输出

<s>[INST]How often does the letter r occur in Mistral?[/INST]To determine how often the letter "r" occurs in the word "Mistral,"we can simply count the instances of "r" in the word.The word "Mistral" is broken down as follows:- M- i- s- t- r- a- lCounting the "r"s, we find that there is only one "r" in "Mistral."Therefore, the letter "r" occurs once in the word "Mistral."
</s>

看来 Mistral 尝试用 CoT 来修复草莓问题🙂

资料

https://mistral.ai/news/september-24-release/

https://artificialanalysis.ai/models/mistral-small

https://huggingface.co/mistralai/Mistral-Small-Instruct-2409

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.rhkb.cn/news/429344.html

如若内容造成侵权/违法违规/事实不符，请联系长河编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

【路径规划】自动泊车的 Simulink 模型

【路径规划】自动泊车的 Simulink 模型

摘要 本文介绍了一个用于自主机器人路径规划和导航的 Simulink 模型&#xff0c;该模型结合了路径跟踪算法&#xff08;如 Pure Pursuit&#xff09;和动态机器人模型&#xff0c;实现了复杂环境中的路径跟随和导航控制。实验结果表明&#xff0c;模型能够在给定路径上精确控制…
阅读更多...
QT快速安装使用指南

QT快速安装使用指南

在Ubuntu 16.04上安装Qt可以通过多种方式进行。以下是使用Qt在线安装程序和apt包管理器的两种常见方法&#xff1a; 方法一&#xff1a;使用Qt在线安装程序 下载Qt在线安装程序 访问Qt官方网站&#xff1a;Try Qt | Develop Applications and Embedded Systems | Qt找到并下载…
阅读更多...
初识ZYNQ——FPGA学习笔记15

初识ZYNQ——FPGA学习笔记15

一、ZYNQ简介 ZYNQ&#xff1a;Zynq-7000 All Programmable SoC&#xff08;APSoC&#xff09;&#xff0c;赛灵思公司&#xff08;AMD Xilinx&#xff09;推出的新一代全可编程片上系统 PS&#xff1a;Processing System&#xff0c;处理系统 PL&#xff1a;Program Logic&…
阅读更多...
Linux：路径末尾加/和不加/的区别

Linux：路径末尾加/和不加/的区别

相关阅读 Linuxhttps://blog.csdn.net/weixin_45791458/category_12234591.html?spm1001.2014.3001.5482 普通文件操作 首先说明这个问题只会出现在目录和符号链接中&#xff0c;因为如果想要索引普通文件但却在路径末尾加/则会出现错误&#xff0c;如例1所示。 # 例1 zhang…
阅读更多...
Zotero(7.0.5)+123云盘同步空间+Z-library=无限存储文献pdf/epub电子书等资料

Zotero(7.0.5)+123云盘同步空间+Z-library=无限存储文献pdf/epub电子书等资料

选择123云盘作为存储介质的原因 原因1&#xff1a; zotero个人免费空间大小&#xff1a;300M&#xff0c;如果zotero云端也保存文献pdf资料则远远不够 原因2&#xff1a; 百度网盘同步文件空间大小&#xff1a;1G123云盘同步文件空间大小&#xff1a;10G 第一台电脑实施步骤…
阅读更多...
Hadoop的一些高频面试题 --- hdfs、mapreduce以及yarn的面试题

Hadoop的一些高频面试题 --- hdfs、mapreduce以及yarn的面试题

文章目录 一、HDFS1、Hadoop的三大组成部分2、本地模式和伪分布模式的区别是什么3、什么是HDFS4、如何单独启动namenode5、hdfs的写入流程6、hdfs的读取流程7、hdfs为什么不能存储小文件8、secondaryNameNode的运行原理9、hadoop集群启动后离开安全模式的条件10、hdfs集群的开机…
阅读更多...
如何导入一个Vue并成功运行

如何导入一个Vue并成功运行

注意1&#xff1a;要确保自己已经成功创建了一个Vue项目&#xff0c;创建项目教程在如何创建Vue项目 注意2&#xff1a;以下操作均在VS Code&#xff0c;教程在VS Code安装教程 一、Vue项目导入VS Code 1.点击文件&#xff0c;然后点击将文件添加到工作区 2. 选择自己的vue项…
阅读更多...
有女朋友后，怎么养成贤内助？为自己找个好伴侣，为孩子找个好妈妈，为母亲找个好儿媳

有女朋友后，怎么养成贤内助？为自己找个好伴侣，为孩子找个好妈妈，为母亲找个好儿媳

有女朋友后&#xff0c;怎么养成贤内助&#xff1f;为自己找个好伴侣&#xff0c;为孩子找个好妈妈&#xff0c;为母亲找个好儿媳 时代背景女生有点作怎么办&#xff1f;大商家族的爱情观 时代背景 一块钱的东西&#xff0c;赋予俩块钱的意义&#xff0c;三块钱卖出去。 用商…
阅读更多...
企业急于采用人工智能，忽视了安全强化

企业急于采用人工智能，忽视了安全强化

对主要云提供商基础设施上托管的资产的安全分析显示&#xff0c;许多公司为了急于构建和部署 AI 应用程序而打开安全漏洞。常见的发现包括对 AI 相关服务使用默认且可能不安全的设置、部署易受攻击的 AI 软件包以及不遵循安全强化指南。 这项分析由 Orca Security 的研究人员进…
阅读更多...
Python爬虫使用实例-umei

Python爬虫使用实例-umei

优美图库 www.umei.cc BV1Ag41137re 1/获取资源 查看网站资源结构 多页&#xff0c;每个item只有一张图 多页&#xff0c;每个item都是一个图集 最大页码 内外层图集均有若干page。 通过尾页按钮确定pageNum&#xff1a; 2/发送请求 response requests.get(urlurl, header…
阅读更多...
蓝桥杯【物联网】零基础到国奖之路:十. OLED

蓝桥杯【物联网】零基础到国奖之路:十. OLED

蓝桥杯【物联网】零基础到国奖之路:十.OLED 第一节 硬件解读第二节 MDK配置 ![在这里插入图片描述](https://i-blog.csdnimg.cn/direct/fa7660b81be9407aa19c603561553db0.png)第三节 代码 第一节 硬件解读 OLED硬件知识: 第二节 MDK配置 第三节 代码 include头文件。 编…
阅读更多...
Vue3 中组件传递 + css 变量的组合

Vue3 中组件传递 + css 变量的组合

文章目录 需求效果如下图所示代码逻辑代码参考 需求 开发一个箭头组件&#xff0c;根据父组件传递的 props 来修改 css 的颜色 效果如下图所示 代码逻辑 代码 父组件&#xff1a; <Arrow color"red" />子组件&#xff1a; <template><div class&…
阅读更多...
VM-Ubantu中使用vscode头文件报错——解决办法

VM-Ubantu中使用vscode头文件报错——解决办法

问题 系统中头文件明明存在但是却报错 解决方法 在报错的文件中点击&#xff0c;shift ctrl p选择Edit Configurations(JSON) 修改文件内容 原文件内容 修改之后的内容 {"configurations": [{"name": "Linux","includePath":…
阅读更多...
https加密原理

https加密原理

以为http的数据都是以明文传送&#xff0c;会有很大的安全问题&#xff0c;所以出现的https协议。https就是在http协议的基础上增加了一个安全层&#xff0c;可以对数据进行加密和解密(例如SSL、TLS等)。 https加密解密的原理&#xff1a;证书非对称加密对称加密 在讲解原理前…
阅读更多...
你了解system V的ipc底层如何设计的吗？消息队列互相通信的原理是什么呢？是否经常将信号量和信号混淆呢？——问题详解

你了解system V的ipc底层如何设计的吗？消息队列互相通信的原理是什么呢？是否经常将信号量和信号混淆呢？——问题详解

前言&#xff1a;本节主要讲解消息队列&#xff0c; 信号量的相关知识。 ——博主主要是以能够理解为目的进行讲解&#xff0c; 所以对于接口的使用或者底层原理很少涉及。 主要的讲解思路就是先讨论消息队列的原理&#xff0c; 提一下接口。 然后讲解ipc的设计——这个设计一些…
阅读更多...
QT打包--windeployqt执行，运行程序提示缺少库

QT打包--windeployqt执行，运行程序提示缺少库

执行windeployqt.exe D:\Data\code\QtCode\Release\RegularExp\RegularExp.exe 生成相应的dll动态库 执行RegularExp.exe&#xff0c;出错&#xff1a;“由于找不到libgcc_s_seh-1.dll…” 找到安装的qt对应的libgcc_s_seh-1.dll拷贝到RegularExp.exe同级目录下&#xff0c; 执…
阅读更多...
Qt 窗口事件机制

Qt 窗口事件机制

在 Qt 开发中&#xff0c;窗口的关闭、隐藏、显示等事件是常见且重要的功能。不同的事件触发条件、处理方式不同&#xff0c;了解和掌握这些事件有助于我们更好地控制窗口行为。本文将详细讲解这些事件的使用方法&#xff0c;并通过代码实例来展示其应用。 1. done(int r) — 关…
阅读更多...
嵌入式C语言自我修养：GNU C编译器扩展语法精讲

嵌入式C语言自我修养：GNU C编译器扩展语法精讲

在Linux内核的源码中&#xff0c;你会发现许多这样的“奇特”代码。它们看起来可能有点陌生&#xff0c;但它们实际上是C语言的一种扩展形式&#xff0c;这种扩展在C语言的标准教材中往往不会提及。这就是为什么你在阅读Linux驱动代码或内核源码时&#xff0c;可能会感到既熟悉…
阅读更多...
写一下线性表

写一下线性表

如果你是c语言, "不会"c, 那么... 把iostream当成stdio.h 把cout当成printf, 不用管啥类型, 变量名字一给输出完事 把cin>>当成scanf, 变量名字一给输入完事 把endl当成\n, 换行. 哦对了, malloc已经不建议使用了, 现在使用new, 把new当作malloc, 把delete当…
阅读更多...
【工具变量】科技金融试点城市DID数据集（2000-2023年）

【工具变量】科技金融试点城市DID数据集（2000-2023年）

时间跨度&#xff1a;2000-2023年数据范围&#xff1a;286个地级市包含指标&#xff1a; year city treat post DID&#xff08;treat*post&#xff09; 样例数据&#xff1a; 包含内容&#xff1a; 全部内容下载链接&#xff1a; 参考文献-pdf格式&#xff1a;https://…
阅读更多...
最新文章
推荐文章

Copyright @ 2022~2023