网站建立软件,湖北公众号定制开发,重庆网站建设咨询,discuz论坛seo设置本教程将引导你在Azure平台完成对 gpt-35-turbo-0613 模型的微调。 关注TechLead#xff0c;分享AI全维度知识。作者拥有10年互联网服务架构、AI产品研发经验、团队管理经验#xff0c;同济本复旦硕#xff0c;复旦机器人智能实验室成员#xff0c;阿里云认证的资深架构师分享AI全维度知识。作者拥有10年互联网服务架构、AI产品研发经验、团队管理经验同济本复旦硕复旦机器人智能实验室成员阿里云认证的资深架构师项目管理专业人士上亿营收AI产品研发负责人 教程介绍
本教程介绍如何执行下列操作
创建示例微调数据集。为资源终结点和 API 密钥创建环境变量。准备样本训练和验证数据集以进行微调。上传训练文件和验证文件进行微调。为 gpt-35-turbo-0613 创建微调作业。部署自定义微调模型。
环境准备 Azure 订阅 - 免费创建订阅。 已在所需的 Azure 订阅中授予对 Azure OpenAI 的访问权限 目前仅应用程序授予对此服务的访问权限。 可以通过在 https://aka.ms/oai/access 上填写表单来申请对 Azure OpenAI 的访问权限。 Python 3.7.1 或更高版本 以下 Python 库json、requests、os、tiktoken、time、openai。 OpenAI Python 库应至少为版本 1.0。 Jupyter Notebook [可进行 gpt-35-turbo-0613 微调的区域]中的 Azure OpenAI 资源。 微调访问需要认知服务 OpenAI 参与者。
设置
Python 库
OpenAI Python 1.x
pip install openai json requests os tiktoken time检索密钥和终结点
若要成功对 Azure OpenAI 发出调用需要一个终结点和一个密钥。
变量名称值ENDPOINT从 Azure 门户检查资源时可在“密钥和终结点”部分中找到此值。 或者可以在“Azure OpenAI Studio”“操场”“代码视图”中找到该值。 示例终结点为https://docs-test-001.openai.azure.com/。API-KEY从 Azure 门户检查资源时可在“密钥和终结点”部分中找到此值。 可以使用 KEY1 或 KEY2。
在 Azure 门户中转到你的资源。 可以在“资源管理”部分找到“终结点和密钥”。 复制终结点和访问密钥因为在对 API 调用进行身份验证时需要这两项。 可以使用 KEY1 或 KEY2。 始终准备好两个密钥可以安全地轮换和重新生成密钥而不会导致服务中断。
环境变量
命令行
setx AZURE_OPENAI_API_KEY REPLACE_WITH_YOUR_KEY_VALUE_HERE setx AZURE_OPENAI_ENDPOINT REPLACE_WITH_YOUR_ENDPOINT_HERE PowerShell
[System.Environment]::SetEnvironmentVariable(AZURE_OPENAI_API_KEY, REPLACE_WITH_YOUR_KEY_VALUE_HERE, User)[System.Environment]::SetEnvironmentVariable(AZURE_OPENAI_ENDPOINT, REPLACE_WITH_YOUR_ENDPOINT_HERE, User)Bash
echo export AZURE_OPENAI_API_KEYREPLACE_WITH_YOUR_KEY_VALUE_HERE /etc/environment
echo export AZURE_OPENAI_ENDPOINTREPLACE_WITH_YOUR_ENDPOINT_HERE /etc/environmentsource /etc/environment创建示例数据集
微调 gpt-35-turbo-0613 需要特殊格式的 JSONL 训练文件。 OpenAI 在其文档中提供了以下示例
{messages: [{role: system, content: Marv is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the capital of France?}, {role: assistant, content: Paris, as if everyone doesnt know that already.}]}
{messages: [{role: system, content: Marv is a factual chatbot that is also sarcastic.}, {role: user, content: Who wrote Romeo and Juliet?}, {role: assistant, content: Oh, just some guy named William Shakespeare. Ever heard of him?}]}
{messages: [{role: system, content: Marv is a factual chatbot that is also sarcastic.}, {role: user, content: How far is the Moon from Earth?}, {role: assistant, content: Around 384,400 kilometers. Give or take a few, like that really matters.}]}对于本示例我们将稍作修改将其更改为
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the capital of France?}, {role: assistant, content: Paris, as if everyone doesnt know that already.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who wrote Romeo and Juliet?}, {role: assistant, content: Oh, just some guy named William Shakespeare. Ever heard of him?}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: How far is the Moon from Earth?}, {role: assistant, content: Around 384,400 kilometers. Give or take a few, like that really matters.}]}尽管这三个示例有助于提供常规格式但如果你想引导自定义微调模型以类似的方式进行响应则需要更多示例。 一般来说你需要至少 50 个高质量示例来开始。然而你完全有可能有一个用例它可能需要 1,000 个高质量的训练示例才能成功。
通常将数据集大小加倍会促使模型质量呈线性增长。 但请记住低质量的示例可能会对性能产生负面影响。 如果你根据大量内部数据来训练模型而不是先修剪数据集以便获得最高质量的示例则最终可能会得到一个性能比预期差得多的模型。
需要创建两个文件training_set.jsonl 和 validation_set.jsonl。
在运行 Jupyter Notebook 的同一目录中创建文件并将以下代码块的内容复制到相应的文件中
training_set.jsonl
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who discovered Antarctica?}, {role: assistant, content: Some chaps named Fabian Gottlieb von Bellingshausen and Mikhail Lazarev, as if they dont teach that in every school!}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the biggest ocean?}, {role: assistant, content: The Pacific Ocean. Its not like its a small pond or anything.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the largest planet?}, {role: assistant, content: Its called Jupiter, you may have heard of it...or not.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who painted the Mona Lisa?}, {role: assistant, content: Oh, some obscure artist named Leonardo da Vinci. Not that he is famous or anything.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the boiling point of water?}, {role: assistant, content: Exciting fact of the day: its 100 degrees Celsius. Life-changing, isnt it?}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who wrote The Odyssey?}, {role: assistant, content: Oh, just some poet named Homer. Nothing special, really.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the largest mammal?}, {role: assistant, content: Blue Whale. Quite a big fish isnt it? Oh wait, its not a fish!}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the highest mountain peak?}, {role: assistant, content: Everest. Brace yourself, its not a walk in the park.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who invented the telephone?}, {role: assistant, content: A guy named Alexander Graham Bell. Bet you didnt guess that one!}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the distance from Earth to the Sun?}, {role: assistant, content: About 93 million miles. Just a quick drive, really.}]}validation_set.jsonl
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the capital of Australia?}, {role: assistant, content: Its Canberra, not Sydney. Shocking, I know!}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who wrote The Great Gatsby?}, {role: assistant, content: Some guy named F. Scott Fitzgerald. Ever heard of him?}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the deepest part of the ocean?}, {role: assistant, content: Mariana Trench, makes your swimming pool seem a bit shallow, doesnt it?}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who composed the Moonlight Sonata?}, {role: assistant, content: Just a little-known guy named Beethoven. No big deal.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What are the colors of the rainbow?}, {role: assistant, content: ROYGBIV. Red, orange, yellow, green, blue, indigo, violet. Not that we learned this in preschool or anything!}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the smallest planet?}, {role: assistant, content: Meet Mercury, the runt of the solar system family.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Who wrote Pride and Prejudice?}, {role: assistant, content: Do the words Jane Austen ring a bell? No? Well, she wrote it.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the largest desert?}, {role: assistant, content: Antarctica. Surprise, surprise! Deserts arent just full of sand, you know.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: What is the longest river?}, {role: assistant, content: The Nile River. Its not like its famous or anything.}]}
{messages: [{role: system, content: Clippy is a factual chatbot that is also sarcastic.}, {role: user, content: Whats the capital of Germany?}, {role: assistant, content: Berlin. Shocking news, right?}]}现在你需要对训练和验证文件运行一些初步检查。
import json# Load the training set
with open(training_set.jsonl, r, encodingutf-8) as f:training_dataset [json.loads(line) for line in f]# Training dataset stats
print(Number of examples in training set:, len(training_dataset))
print(First example in training set:)
for message in training_dataset[0][messages]:print(message)# Load the validation set
with open(validation_set.jsonl, r, encodingutf-8) as f:validation_dataset [json.loads(line) for line in f]# Validation dataset stats
print(\nNumber of examples in validation set:, len(validation_dataset))
print(First example in validation set:)
for message in validation_dataset[0][messages]:print(message)输出
Number of examples in training set: 10
First example in training set:
{role: system, content: Clippy is a factual chatbot that is also sarcastic.}
{role: user, content: Who discovered America?}
{role: assistant, content: Some chap named Christopher Columbus, as if they dont teach that in every school!}Number of examples in validation set: 10
First example in validation set:
{role: system, content: Clippy is a factual chatbot that is also sarcastic.}
{role: user, content: Whats the capital of Australia?}
{role: assistant, content: Its Canberra, not Sydney. Shocking, I know!}在本例中我们只有 10 个训练示例和 10 个验证示例因此虽然这将演示微调模型的基本机制但示例数量不太可能足以产生持续明显的影响。
现在可以使用 tiktoken 库从 OpenAI 运行一些额外的代码来验证令牌计数。 各个示例需要保持在 gpt-35-turbo-0613 模型的 4096 个令牌的输入令牌限制内。
import json
import tiktoken
import numpy as np
from collections import defaultdictencoding tiktoken.get_encoding(cl100k_base) # default encoding used by gpt-4, turbo, and text-embedding-ada-002 modelsdef num_tokens_from_messages(messages, tokens_per_message3, tokens_per_name1):num_tokens 0for message in messages:num_tokens tokens_per_messagefor key, value in message.items():num_tokens len(encoding.encode(value))if key name:num_tokens tokens_per_namenum_tokens 3return num_tokensdef num_assistant_tokens_from_messages(messages):num_tokens 0for message in messages:if message[role] assistant:num_tokens len(encoding.encode(message[content]))return num_tokensdef print_distribution(values, name):print(f\n#### Distribution of {name}:)print(fmin / max: {min(values)}, {max(values)})print(fmean / median: {np.mean(values)}, {np.median(values)})print(fp5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)})files [training_set.jsonl, validation_set.jsonl]for file in files:print(fProcessing file: {file})with open(file, r, encodingutf-8) as f:dataset [json.loads(line) for line in f]total_tokens []assistant_tokens []for ex in dataset:messages ex.get(messages, {})total_tokens.append(num_tokens_from_messages(messages))assistant_tokens.append(num_assistant_tokens_from_messages(messages))print_distribution(total_tokens, total tokens)print_distribution(assistant_tokens, assistant tokens)print(* * 50)输出
Processing file: training_set.jsonl#### Distribution of total tokens:
min / max: 47, 57
mean / median: 50.8, 50.0
p5 / p95: 47.9, 55.2#### Distribution of assistant tokens:
min / max: 13, 21
mean / median: 16.3, 15.5
p5 / p95: 13.0, 20.1
**************************************************
Processing file: validation_set.jsonl#### Distribution of total tokens:
min / max: 43, 65
mean / median: 51.4, 49.0
p5 / p95: 45.7, 56.9#### Distribution of assistant tokens:
min / max: 8, 29
mean / median: 15.9, 13.5
p5 / p95: 11.6, 20.9
**************************************************上传微调文件
OpenAI Python 1.x
# Upload fine-tuning filesimport os
from openai import AzureOpenAIclient AzureOpenAI(azure_endpoint os.getenv(AZURE_OPENAI_ENDPOINT), api_keyos.getenv(AZURE_OPENAI_KEY), api_version2023-12-01-preview # This API version or later is required to access fine-tuning for turbo/babbage-002/davinci-002
)training_file_name training_set.jsonl
validation_file_name validation_set.jsonl# Upload the training and validation dataset files to Azure OpenAI with the SDK.training_response client.files.create(fileopen(training_file_name, rb), purposefine-tune
)
training_file_id training_response.idvalidation_response client.files.create(fileopen(validation_file_name, rb), purposefine-tune
)
validation_file_id validation_response.idprint(Training file ID:, training_file_id)
print(Validation file ID:, validation_file_id)输出
Training file ID: file-9ace76cb11f54fdd8358af27abf4a3ea
Validation file ID: file-70a3f525ed774e78a77994d7a1698c4b开始微调
现在微调文件已成功上传可以提交微调训练作业
OpenAI Python 1.x
response client.fine_tuning.jobs.create(training_filetraining_file_id,validation_filevalidation_file_id,modelgpt-35-turbo-0613, # Enter base model name. Note that in Azure OpenAI the model name contains dashes and cannot contain dot/period characters.
)job_id response.id# You can use the job ID to monitor the status of the fine-tuning job.
# The fine-tuning job will take some time to start and complete.print(Job ID:, response.id)
print(Status:, response.id)
print(response.model_dump_json(indent2))输出
Job ID: ftjob-40e78bc022034229a6e3a222c927651c
Status: pending
{hyperparameters: {n_epochs: 2},status: pending,model: gpt-35-turbo-0613,training_file: file-90ac5d43102f4d42a3477fd30053c758,validation_file: file-e21aad7dddbc4ddc98ba35c790a016e5,id: ftjob-40e78bc022034229a6e3a222c927651c,created_at: 1697156464,updated_at: 1697156464,object: fine_tuning.job
}跟踪训练作业状态
如果想轮询训练作业状态直至其完成可以运行
OpenAI Python 1.x
# Track training statusfrom IPython.display import clear_output
import timestart_time time.time()# Get the status of our fine-tuning job.
response client.fine_tuning.jobs.retrieve(job_id)status response.status# If the job isnt done yet, poll it every 10 seconds.
while status not in [succeeded, failed]:time.sleep(10)response client.fine_tuning.jobs.retrieve(job_id)print(response.model_dump_json(indent2))print(Elapsed time: {} minutes {} seconds.format(int((time.time() - start_time) // 60), int((time.time() - start_time) % 60)))status response.statusprint(fStatus: {status})clear_output(waitTrue)print(fFine-tuning job {job_id} finished with status: {status})# List all fine-tuning jobs for this resource.
print(Checking other fine-tune jobs for this resource.)
response client.fine_tuning.jobs.list()
print(fFound {len(response.data)} fine-tune jobs.)输出
{hyperparameters: {n_epochs: 2},status: running,model: gpt-35-turbo-0613,training_file: file-9ace76cb11f54fdd8358af27abf4a3ea,validation_file: file-70a3f525ed774e78a77994d7a1698c4b,id: ftjob-0f4191f0c59a4256b7a797a3d9eed219,created_at: 1695307968,updated_at: 1695310376,object: fine_tuning.job
}
Elapsed time: 40 minutes 45 seconds
Status: running需要一个多小时才能完成训练的情况并不罕见。 训练完成后输出消息将更改为
Fine-tuning job ftjob-b044a9d3cf9c4228b5d393567f693b83 finished with status: succeeded
Checking other fine-tuning jobs for this resource.
Found 2 fine-tune jobs.若要获取完整结果请运行以下命令
OpenAI Python 1.x
#Retrieve fine_tuned_model nameresponse client.fine_tuning.jobs.retrieve(job_id)print(response.model_dump_json(indent2))
fine_tuned_model response.fine_tuned_model部署微调的模型
与本教程中前面的 Python SDK 命令不同引入配额功能后模型部署必须使用 [REST API]完成这需要单独的授权、不同的 API 路径和不同的 API 版本。
或者可以使用任何其他常见部署方法例如 Azure OpenAI Studio 或 [Azure CLI]来部署微调模型。
variable定义token可通过多种方式生成授权令牌。 初始测试的最简单方法是从 Azure 门户启动 Cloud Shell。 然后运行 az account get-access-token。 可以将此令牌用作 API 测试的临时授权令牌。 建议将其存储在新的环境变量中订阅关联的 Azure OpenAI 资源的订阅 IDresource_groupAzure OpenAI 资源的资源组名称resource_nameAzure OpenAI 资源名称model_deployment_name新微调模型部署的自定义名称。 这是在进行聊天补全调用时将在代码中引用的名称。fine_tuned_model请从上一步的微调作业结果中检索此值。 该字符串类似于 gpt-35-turbo-0613.ft-b044a9d3cf9c4228b5d393567f693b83。 需要将该值添加到 deploy_data json。
import json
import requeststoken os.getenv(TEMP_AUTH_TOKEN)
subscription YOUR_SUBSCRIPTION_ID
resource_group YOUR_RESOURCE_GROUP_NAME
resource_name YOUR_AZURE_OPENAI_RESOURCE_NAME
model_deployment_name YOUR_CUSTOM_MODEL_DEPLOYMENT_NAMEdeploy_params {api-version: 2023-05-01}
deploy_headers {Authorization: Bearer {}.format(token), Content-Type: application/json}deploy_data {sku: {name: standard, capacity: 1}, properties: {model: {format: OpenAI,name: YOUR_FINE_TUNED_MODEL, #retrieve this value from the previous call, it will look like gpt-35-turbo-0613.ft-b044a9d3cf9c4228b5d393567f693b83version: 1}}
}
deploy_data json.dumps(deploy_data)request_url fhttps://management.azure.com/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.CognitiveServices/accounts/{resource_name}/deployments/{model_deployment_name}print(Creating a new deployment...)r requests.put(request_url, paramsdeploy_params, headersdeploy_headers, datadeploy_data)print(r)
print(r.reason)
print(r.json())可以在 Azure OpenAI Studio 中检查部署进度 在处理部署微调模型时此过程需要一些时间才能完成的情况并不罕见。
使用已部署的自定义模型
部署微调后的模型后可以使用该模型就像使用 Azure OpenAI Studio 的聊天平台中的任何其他已部署模型一样或通过聊天完成 API 中来使用它。 例如可以向已部署的模型发送聊天完成调用如以下 Python 示例中所示。 可以继续对自定义模型使用相同的参数例如温度和 max_tokens就像对其他已部署的模型一样。
OpenAI Python 1.x
import os
from openai import AzureOpenAIclient AzureOpenAI(azure_endpoint os.getenv(AZURE_OPENAI_ENDPOINT), api_keyos.getenv(AZURE_OPENAI_KEY), api_version2023-05-15
)response client.chat.completions.create(modelgpt-35-turbo-ft, # model Custom deployment name you chose for your fine-tuning modelmessages[{role: system, content: You are a helpful assistant.},{role: user, content: Does Azure OpenAI support customer managed keys?},{role: assistant, content: Yes, customer managed keys are supported by Azure OpenAI.},{role: user, content: Do other Azure AI services support this too?}]
)print(response.choices[0].message.content)删除部署
与其他类型的 Azure OpenAI 模型不同微调/自定义模型在部署后会产生关联的每小时托管费用。 强烈建议你在完成本教程并针对微调后的模型测试了一些聊天完成调用后删除模型部署。
删除部署不会对模型本身产生任何影响因此你可以随时重新部署为本教程训练的微调模型。
可以通过 [REST API]、[Azure CLI]或其他支持的部署方法删除 Azure OpenAI Studio 中的部署。 关注TechLead分享AI全维度知识。作者拥有10年互联网服务架构、AI产品研发经验、团队管理经验同济本复旦硕复旦机器人智能实验室成员阿里云认证的资深架构师项目管理专业人士上亿营收AI产品研发负责人
