模型推理服务人工智能后端大模型MLOpsLLMOps【免费下载链接】BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!项目地址https://gitcode.com/gh_mirrors/be/BentoML点击查看免费下载本篇指南基于 BentoML 官方入门文档 docs/source/get-started/hello-world.rst完整演示了从零构建一个文本摘要Text Summarization推理服务的最小可行流程搭建环境、编写 Service、本地启动服务并用多种方式调用 API。读完本篇你将掌握 BentoML 的核心抽象bentoml.service/bentoml.api的用法、bentoml serve本地开发工作流以及仓库内对应的端到端测试用例与真实示例配置为后续打包镜像、部署云端打下基础。教程目标与前置知识本教程要完成三件事搭建 BentoML 开发环境克隆示例仓库、创建虚拟环境、安装bentoml、torch、transformers依赖编写一个 BentoML Service在service.py中定义一个Summarization类用bentoml.service装饰内部加载 Hugging Face 的summarizationpipeline本地启动并调用服务bentoml serve拉起 HTTP 服务通过 cURL、Python 客户端和 Swagger UI 三种方式调用/summarize端点。在 BentoML 的体系里Service 是可部署、可伸缩的最小服务单元它由bentoml.service装饰的 Python 类定义负责管理模型状态及其生命周期并通过bentoml.api装饰的方法对外暴露一个或多个 HTTP API。这正是本教程的核心概念源码实现位于 src/_bentoml_sdk/decorators.py。官方示例源码位于bentoml/quickstartGitHub 仓库在当前 BentoML 仓库中与之等价的可运行示例被放在 tests/e2e/fixtures/quickstart 目录下含service.py、bentofile.yaml、requirements.txt配套的端到端测试见 tests/e2e/bento_new_sdk/test_quickstart.py。第一步搭建开发环境1. 获取示例项目git clone https://github.com/bentoml/quickstart.git cd quickstart2. 创建并激活虚拟环境推荐使用虚拟环境隔离依赖# Mac/Linux python3 -m venv quickstart source quickstart/bin/activate# Windows python -m venv quickstart quickstart\Scripts\activate如果你不想在本地搭建环境可以直接跳到 BentoCloud 云端部署文档。3. 安装依赖# 推荐 Python 3.11 pip install bentoml torch transformers仓库内的示例项目对依赖做了版本约束见 tests/e2e/fixtures/quickstart/requirements.txtbentoml torch transformers4.54.0安装完成后可以用bentoml --version验证 CLI 是否可用。bentoml的 CLI 入口实现在 src/bentoml_cli/cli.py。第二步编写 BentoML Service在项目根目录创建service.py代码如下from __future__ import annotations import bentoml with bentoml.importing(): from transformers import pipeline EXAMPLE_INPUT Breaking News: In an astonishing turn of events, the small town of Willow Creek has been taken by storm as local resident Jerry Thompsons cat, Whiskers, performed what witnesses are calling a miraculous and gravity-defying leap. Eyewitnesses report that Whiskers, an otherwise unremarkable tabby cat, jumped a record-breaking 20 feet into the air to catch a fly. The event, which took place in Thompsons backyard, is now being investigated by scientists for potential breaches in the laws of physics. Local authorities are considering a town festival to celebrate what is being hailed as The Leap of the Century. bentoml.service class Summarization: def __init__(self) - None: self.pipeline pipeline(summarization) bentoml.api def summarize(self, text: str EXAMPLE_INPUT) - str: result self.pipeline(text) return fHello world! Heres your summary: {result[0][summary_text]}代码逐段解读bentoml.service装饰器将Summarization类声明为一个 BentoML Service__init__中完成模型与 pipeline 的初始化。注意初始化只发生一次之后每个请求都会复用同一个 pipeline 实例避免重复加载模型带来的开销Service 可以持有状态并管理其生命周期包括模型实例、连接池等资源Service 通过bentoml.api装饰的方法对外暴露 API。bentoml.api装饰器将summarize方法声明为 HTTP 端点方法签名中的参数类型注解str会用于自动生成 API 的输入输出契约text: str EXAMPLE_INPUT提供了默认示例输入方便在 Swagger UI 中直接点击测试从源码 src/_bentoml_sdk/decorators.py 可以看到bentoml.api还支持route自定义路由路径、name、input_spec/output_specPydantic 模型、batchable自适应批处理、batch_dim、max_batch_size默认 100、max_latency_ms默认 60000等参数本教程使用最简形式其余参数取默认值。bentoml.importing()上下文管理器用于包裹仅运行时需要的依赖导入它的实现位于 src/_bentoml_impl/loader.py在非服务运行时比如模型注册、Bento 构建阶段如果transformers尚未安装它会捕获ImportError并跳过导入只记录一条 info 日志而在服务真正运行的 worker 中server_context.worker_index非空异常会被原样抛出确保服务启动时依赖确实可用这样既允许在没有完整推理依赖的环境里加载、构建 Service又保证了线上运行时的依赖完整性。仓库中的生产级变体当前仓库的 fixtures 目录里有一个更贴近生产实践的版本 tests/e2e/fixtures/quickstart/service.pyfrom __future__ import annotations # I001 import bentoml bentoml.service(resources{cpu: 4}) class Summarization: def __init__(self) - None: import torch from transformers import pipeline device cuda if torch.cuda.is_available() else cpu self.pipeline pipeline(summarization, devicedevice) bentoml.api(batchableTrue) def summarize(self, texts: list[str]) - list[str]: results self.pipeline(texts) return [item[summary_text] for item in results]对比可见生产级写法增加了三处关键能力特性入门写法生产级写法资源声明无resources{cpu: 4}声明 CPU 配额硬件适配默认 CPU自动检测cuda/cpu并指定 device批处理单条输入batchableTrue输入输出均为list[str]由 BentoML 自动聚合请求为 batch其中batchableTrue对应的自适应批处理机制正是官方文档 adaptive-batching.rst 的主题。第三步本地启动服务在项目根目录运行$ bentoml serve成功启动后你会看到类似输出2024-02-02T07:16:140000 [WARNING] [cli] Converting Summarization to lowercase: summarization. 2024-02-02T07:16:150000 [INFO] [cli] Starting production HTTP BentoServer from service:Summarization listening on http://localhost:3000 (Press CTRLC to quit)服务默认监听http://localhost:3000。第一条 WARNING 提示 Service 类名Summarization会被规范化为小写summarization用于服务标识。bentoml serve命令详解bentoml serve的实现位于 src/bentoml_cli/serve.py核心参数如下参数说明默认值bento位置参数服务目标可以是Service 实例导入路径如service.py:Summarization、本地 Bento 存储中的 Tag如summarization:latest、包含bentofile.yaml的目录、或已构建 Bento 的路径.当前目录-p, --portREST API 服务监听端口3000可用环境变量BENTOML_PORT覆盖--host绑定主机生产模式绑定所有接口开发模式默认127.0.0.1可用BENTOML_HOST覆盖--development开发模式单 worker、默认 host 127.0.0.1False--reload检测代码变化自动重启服务False--working-dir指定 Service 源码所在目录当前目录--timeoutAPI 服务器与 runner 的超时秒数无可用BENTOML_TIMEOUT覆盖--api-workersAPI server worker 数量生产模式默认等于 CPU 核数由BENTOML_API_WORKERS覆盖注意--production参数已废弃生产模式默认开启如需开发模式请显式传--development见 src/bentoml_cli/serve.py 中的 DeprecationWarning 逻辑。--reload在开发阶段非常实用它会监听--working-dir下代码变化以及本地模型存储变化并自动重启服务。第四步调用/summarize端点服务启动后可以通过三种方式调用summarizeAPI。方式一cURLcurl -X POST \ http://localhost:3000/summarize \ -H accept: text/plain \ -H Content-Type: application/json \ -d { text: Breaking News: In an astonishing turn of events, the small town of Willow Creek has been taken by storm as local resident Jerry Thompson\s cat, Whiskers, performed what witnesses are calling a \miraculous and gravity-defying leap.\ Eyewitnesses report that Whiskers, an otherwise unremarkable tabby cat, jumped a record-breaking 20 feet into the air to catch a fly. The event, which took place in Thompson\s backyard, is now being investigated by scientists for potential breaches in the laws of physics. Local authorities are considering a town festival to celebrate what is being hailed as \The Leap of the Century. }方式二Python 客户端BentoML 提供官方同步 HTTP 客户端封装了请求序列化、连接管理与类型转换import bentoml with bentoml.SyncHTTPClient(http://localhost:3000) as client: result client.summarize( textBreaking News: In an astonishing turn of events, the small town of Willow Creek has been taken by storm as local resident Jerry Thompsons cat, Whiskers, performed what witnesses are calling a miraculous and gravity-defying leap. Eyewitnesses report that Whiskers, an otherwise unremarkable tabby cat, jumped a record-breaking 20 feet into the air to catch a fly. The event, which took place in Thompsons backyard, is now being investigated by scientists for potential breaches in the laws of physics. Local authorities are considering a town festival to celebrate what is being hailed as The Leap of the Century. ) print(result)SyncHTTPClient会根据服务端自动生成的 API 契约把client.summarize(text...)映射为对/summarize端点的调用text参数自动 JSON 序列化返回值自动解析为字符串。客户端实现在 src/_bentoml_impl/client 与 src/bentoml/_internal/client 目录下。方式三Swagger UI浏览器访问http://localhost:3000滚动到Service APIs区域点击Try it out在Request body输入框中填入你的 prompt点击Execute即可在线调试接口Swagger UI 界面由 BentoML 基于 FastAPI 自动生成——从源码 src/_bentoml_sdk/decorators.py 可以看到bentoml.asgi_app会将 ASGI 应用如 FastAPI挂载到 Service 上因此 Service 天然获得完整的 OpenAPI 文档与交互式调试能力。预期输出无论用哪种方式调用返回结果大致如下Hello world! Heres your summary: Whiskers, an otherwise unremarkable tabby cat, jumped a record-breaking 20 feet into the air to catch a fly . The event is now being investigated by scientists for potential breaches in the laws of physics . Local authorities considering a town festival to celebrate what is being hailed as The Leap of the Century从源码验证这个教程真的可运行当前仓库提供了与本教程完全对应的端到端测试 tests/e2e/bento_new_sdk/test_quickstart.py从四个维度验证了上述流程test_async_serve_and_prediction用bentoml.serve在指定端口拉起 quickstart 示例再通过bentoml.SyncHTTPClient与bentoml.AsyncHTTPClient分别调用summarize断言摘要结果包含 Whiskers同时验证了application/vnd.bentomlpickle恶意反序列化负载会被拒绝返回 415体现服务端输入校验能力test_local_prediction不启动服务器直接用bentoml.load(...)()实例化 Service 并调用方法验证本地直调开发模式test_build_and_predictionbentoml.build(service.py:Summarization)构建 Bento 后再次启动服务并预测验证构建产物 → 服务化的完整闭环。配套的构建配置文件 tests/e2e/fixtures/quickstart/bentofile.yaml 展示了标准 Bento 构建声明service: service:Summarization labels: project: quickstart stage: dev include: - service.py python: packages: - torch - transformers docker: python_version: 3.10service字段指定 Service 的导入路径include限定打入 Bento 的文件python.packages声明运行期依赖构建时 BentoML 会自动解析docker.python_version指定容器基础镜像的 Python 版本。下一步进阶完成本教程后你已经掌握了 BentoML 服务化的最小闭环。官方文档建议的进阶路线批处理与性能优化学习自适应批处理adaptive batching把单条 API 升级为批量处理接口见 adaptive-batching.rst自定义模型加载与生命周期管理深入理解模型存储、加载与管理机制相关源码位于 src/bentoml/_internal/models/model.py 和 src/bentoml/_internal/bento/bento.py打包为 Docker 镜像用bentoml containerize将 Service 与依赖打包成可移植镜像见 packaging-for-deployment.rst部署到云端将 Bento 部署到 BentoCloud 或你自己的云环境见 cloud-deployment.rst。从本地 Hello World到生产级推理服务BentoML 的核心思想始终如一用 Python 类定义服务用装饰器声明 API其余序列化、路由、文档、批处理、容器化交给框架。赞分享模型推理服务人工智能后端大模型MLOpsLLMOps【免费下载链接】BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!项目地址https://gitcode.com/gh_mirrors/be/BentoML点击查看免费下载相关推荐BentoML Quickstart构建并服务一个 Hugging Face 文本摘要推理服务BentoML Quickstart构建并服务一个 Hugging Face 文本摘要推理服务 本文以 BentoML 仓库自带的 quickstart 端到模型推理服务人工智能后端大模型MLOpsLLMOpsHugging Face课程使用mT5模型实现多语言文本摘要Hugging Face课程使用mT5模型实现多语言文本摘要 文本摘要任务概述 文本摘要是自然语言处理 NLP 中的一项重要任务旨在将长篇文档压缩为简洁的摘文档教程人工智能NLP深度学习3行代码搞定将Wan2.1-Fun-14B-Control模型封装为高性能API服务3行代码搞定将Wan2.1 Fun 14B Control模型封装为高性能API服务 你还在为视频生成模型部署繁琐、调用复杂而头疼吗47GB的超大模型如何在大模型计算机视觉多模态上一篇Torrentio在流媒体海洋中导航的智能导航系统下一篇gh-stack AI Agent用gh skill让Copilot帮你创建和管理Stacked PR创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
