六安论坛网站,建设企业营销型网站,个人优秀网页设计图片,wordpress 网站投票Gemini 是 Google DeepMind 开发的多模态大语言模型家族#xff0c;作为 LaMDA 和 PaLM 2 的后继者。由 Gemini Ultra、Gemini Pro 和 Gemini Nano 组成#xff0c;于 2023 年 12 月 6 日发布#xff0c;定位为 OpenAI 的竞争者 GPT-4。 本教程演示如何使用 Gemini API 创建…Gemini 是 Google DeepMind 开发的多模态大语言模型家族作为 LaMDA 和 PaLM 2 的后继者。由 Gemini Ultra、Gemini Pro 和 Gemini Nano 组成于 2023 年 12 月 6 日发布定位为 OpenAI 的竞争者 GPT-4。 本教程演示如何使用 Gemini API 创建嵌入并将其存储在 Elasticsearch 中。 Elasticsearch 将使我们能够执行向量搜索 (Knn) 来查找相似的文档。 准备 Elasticsearch 及 Kibana
如果你还没有安装好自己的 Elasticsearch 及 Kibana 的话请参阅如下的文章来进行安装 如何在 LinuxMacOS 及 Windows 上进行安装 Elasticsearch Kibana如何在 LinuxMacOS 及 Windows 上安装 Elastic 栈中的 Kibana
在安装的时候请参照 Elastic Stack 8.x 的文章来进行安装。 Gemini 开发者 key
你可以参考文章 来申请一个免费的 key 供下面的开发。你也可以直接去地址进行申请。 设置环境变量
我们在 termnial 中打入如下的命令来设置环境变量
export ES_USERelastic
export ES_PASSWORD-M3aD_m3MHCZNYyJi_V2
export GOOGLE_API_KEYYourGoogleAPIkey 拷贝 Elasticsearch 证书
我们把 Elasticsearch 的证书拷贝到当前的目录下
$ pwd
/Users/liuxg/python/elser
$ cp ~/elastic/elasticsearch-8.12.0/config/certs/http_ca.crt . 安装 Python 依赖包
pip3 install -q -U google-generativeai elasticsearch 应用设计
我们在当前的工作目录下打入命令
jupyter notebook 导入包及环境变量
import google.generativeai as genai
import google.ai.generativelanguage as glm
from elasticsearch import Elasticsearch, helpers
from dotenv import load_dotenv
import osload_dotenv()GOOGLE_API_KEY os.getenv(GOOGLE_API_KEY)
ES_USER os.getenv(ES_USER)
ES_PASSWORD os.getenv(ES_PASSWORD)
elastic_index_namegemini-demo 连接到 Elasticsearch
url fhttps://{ES_USER}:{ES_PASSWORD}192.168.0.3:9200es Elasticsearch(hosts[url], ca_certs ./http_ca.crt, verify_certs True
)
print(es.info()) 上面显示我们的 es 连接是成功的。 删除索引
if(es.indices.exists(indexelastic_index_name)):print(The index has already existed, going to remove it)es.options(ignore_status404).indices.delete(indexelastic_index_name) 使用 Elasticsearch 索引文档
生成一个 title 为 “Beijing” 文档
genai.configure(api_keyGOOGLE_API_KEY)title Beijing
sample_text (Beijing is the capital of China and the center of Chinese politics, culture, and economy. This city has a long history with many ancient buildings and cultural heritage. Beijing is renowned as a cultural city in China, boasting numerous museums, art galleries, and historical landmarks. Additionally, as a modern metropolis, Beijing is a thriving business center with modern architecture and advanced transportation systems. It serves as the seat of the Chinese government, where significant decisions and events often take place. Overall, Beijing holds a crucial position in China, serving as both a preserver of traditional culture and a representative of modern development.)model models/embedding-001
embedding genai.embed_content(modelmodel,contentsample_text,task_typeretrieval_document,titletitle)doc {text : sample_text,text_embedding : embedding[embedding]
}resp es.index(indexelastic_index_name, documentdoc)print(resp)
生成一个 title 为 “Shanghai” 的文档
title Shanghai
sample_text (Shanghai is one of Chinas largest cities and a significant hub for economy, finance, and trade. This modern city is located in the eastern part of China and serves as an international metropolis. The bustling streets, skyscrapers, and modern architecture in Shanghai showcase the citys prosperity and development. As one of Chinas economic engines, Shanghai is home to the headquarters of many international companies and various financial institutions. It is also a crucial trading port, connecting with destinations worldwide. Additionally, Shanghai boasts a rich cultural scene, including art galleries, theaters, and historical landmarks. In summary, Shanghai is a vibrant, modern city with international influence.)model models/embedding-001
embedding genai.embed_content(modelmodel,contentsample_text,task_typeretrieval_document,titletitle)doc {text : sample_text,text_embedding : embedding[embedding]
}resp es.index(indexelastic_index_name, documentdoc)print(resp)
我们可以在 Kibana 中进行查看 使用 Elasticsearch 来搜索文档
def search(question):print(\n\nQuestion: , question)embedding genai.embed_content(modelmodel,contentquestion,task_typeretrieval_query)resp es.search(index elastic_index_name,knn{field: text_embedding,query_vector: embedding[embedding],k: 10,num_candidates: 100})for result in resp[hits][hits]:pretty_output (f\n\nID: {result[_id]}\n\nText: {result[_source][text]})print(pretty_output)
search(How do you describe Beijing?) search(What is Shanghai like?) 从上面的输出中我们可以看出来当搜索的句子和文章更为接近时相关的文档就会排在第一的位置。紧接着的是次之相关的文档。
search(which city is the capital of China?) search(the economy engine in China) 最后源码在位置可以进行下载https://github.com/liu-xiao-guo/semantic_search_es/blob/main/vector-search-using-gemini-elastic.ipynb
