from onprem import LLM
import os
llm = LLM(n_gpu_layers=35)Use Prompts to Solve Problems
In these examples of OnPrem.LLM, we will accelerate inference using a GPU. We use an NVIDIA Titan V GPU with a modest 12GB of VRAM. For GPU acceleration, make sure you installed llama-cpp-python with CUBLAS support as described here.
After that, you just need to supply the n_gpu_layers argument to LLM for GPU-accelerated responses.
Setup the LLM instance
Information Extraction
prompt = """Extract the names of people in the supplied sentences. Here is an example:
Sentence: James Gandolfini and Paul Newman were great actors.
People:
James Gandolfini, Paul Newman
Sentence:
I like Cillian Murphy's acting. Florence Pugh is great, too.
People:"""
saved_output = llm.prompt(prompt)
Cillian Murphy, Florence PughA more complicated example of Information Extraction:
prompt = """Extract the Name, Position, and Company from the following sentences. Here are some examples.
Text: Alan F. Estevez serves as the Under Secretary of Commerce for Industry and Security. As Under Secretary, Mr. Estevez leads
the Bureau of Industry and Security, which advances U.S. national security, foreign policy, and economic objectives by ensuring an
effective export control and treaty compliance system and promoting U.S. strategic technology leadership.
Name: Alan F. Estevez | Position: Under Secretary | Company: Bureau of Industry and Security
###
Text: Pichai Sundararajan (born June 10, 1972[3][4][5]), better known as Sundar Pichai (/ˈsʊndɑːr pɪˈtʃaɪ/), is an Indian-born American
business executive.[6][7] He is the chief executive officer (CEO) of Alphabet Inc. and its subsidiary Google.[8]
Name: Sundar Pichai | Position: CEO | Company: Google
###
Text: Norton Allan Schwartz (born December 14, 1951)[1] is a retired United States Air Force general[2] who served as the 19th Chief of Staff of the
Air Force from August 12, 2008, until his retirement in 2012.[3] He previously served as commander, United States Transportation Command from
September 2005 to August 2008. He is currently the president of the Institute for Defense Analyses, serving since January 2, 2020.[4]
"""
saved_output = llm.prompt(prompt)Name: Norton Schwartz | Position: President| Company: Institute for Defense AnalysesGrammar Correction
prompt = """Correct the grammar and spelling in the supplied sentences. Here are some examples.
[Sentence]:
I love goin to the beach.
[Correction]: I love going to the beach.
[Sentence]:
Let me hav it!
[Correction]: Let me have it!
[Sentence]:
It have too many drawbacks.
[Correction]: It has too many drawbacks.
[Sentence]:
I do not wan to go
[Correction]:"""
saved_output = llm.prompt(prompt) I don't want to go.Classification
prompt = """Classify each sentence as either positive, negative, or neutral. Here are some examples.
[Sentence]: I love going to the beach.
[[Classification]: Positive
[Sentence]: It is 10am right now.
[Classification]: Neutral
[Sentence]: I just got fired from my job.
[Classification]: Negative
[Sentence]: The reactivity of your team has been amazing, thanks!
[Classification]:"""
saved_output = llm.prompt(prompt) PositiveParaphrasing
prompt = """Paraphrase the following text delimited by triple backticks.
```After a war lasting 20 years, following the decision taken first by President Trump and then by President Biden to withdraw American troops, Kabul, the capital of Afghanistan, fell within a few hours to the Taliban, without resistance.```
"""
saved_output = llm.prompt(prompt)Paraphrase: Following a prolonged war in Afghanistan that lasted for 20 years, when both U.S. Presidents Trump and Biden decided to withdraw American troops from the country, Kabul - the capital city of Afghanistan - fell within hours into the hands of the Taliban without any resistance.Few-Shot Answer Extraction
prompt = """Answer the Question based on the Context. Here are some examples.
[Context]:
NLP Cloud was founded in 2021 when the team realized there was no easy way to reliably leverage Natural Language Processing in production.
Question: When was NLP Cloud founded?
[Answer]:
2021
###
[Context]:
NLP Cloud developed their API by mid-2020 and they added many pre-trained open-source models since then.
[Question]:
What did NLP Cloud develop?
[Answer]:
API
###
[Context]:
All plans can be stopped anytime. You only pay for the time you used the service. In case of a downgrade, you will get a discount on your next invoice.
[Question]:
When can plans be stopped?
[Answer]:
Anytime
###
[Context]:
The main challenge with GPT-J is memory consumption. Using a GPU plan is recommended.
[Question]:
Which plan is recommended for GPT-J?
Answer:"""
saved_output = llm.prompt(prompt)
GPU PlanGenerating Product Descriptions
prompt = """Generate a Sentence from the Keywords. Here are some examples.
[Keywords]:
shoes, women, $59
[Sentence]:
Beautiful shoes for women at the price of $59.
###
[Keywords]:
trousers, men, $69
[Sentence]:
Modern trousers for men, for $69 only.
###
[Keywords]:
gloves, winter, $19
[Sentence]:
Amazingly hot gloves for cold winters, at $19.
###
[Keywords]:
t-shirt, men, $39
[Sentence]:"""
saved_output = llm.prompt(prompt)
A stylish and comfortable t-shirt for men, for just $39.Tweet Generation
prompt = """Generate a tweet based on the supplied Keyword. Here are some examples.
[Keyword]:
markets
[Tweet]:
Take feedback from nature and markets, not from people
###
[Keyword]:
children
[Tweet]:
Maybe we die so we can come back as children.
###
[Keyword]:
startups
[Tweet]:
Startups should not worry about how to put out fires, they should worry about how to start them.
###
[Keyword]:
climate change
[Tweet]:"""
saved_output = llm.prompt(prompt)
The climate is changing, but our actions can save it from catastrophe.
###Drafting Emails
prompt = """Generate an email introducing Tesla to shareholders."""
saved_output = llm.prompt(prompt)
Subject: Introducing Tesla, the future of transportation
Dear Shareholder,
I am writing to introduce you to Tesla, the leading manufacturer of electric vehicles (EVs) and energy storage products.
Tesla has been at the forefront of innovating EV technology since its inception in 2003. With a history of creating breakthrough products such as the Tesla Model S, Tesla Powerwall, and Solar Roof, it is clear that Tesla's mission is to create a better future through sustainable transportation and energy solutions.
Tesla has experienced rapid growth in recent years, with its stock price increasing by 475% since January 2016. This growth can be attributed to several factors, including the increase in global interest in sustainability and environmental issues, as well as Tesla's continuous innovation and success in the EV market.
Overall, Tesla is a company that not only produces cutting-edge products but also has a strong commitment to sustainability and innovation. As a shareholder of Tesla, I encourage you to take advantage of this opportunity to invest in the future of transportation and energy storage solutions with Tesla.
Thank you for your continued support of Tesla.Talk to Your Documents
Note: Although we are using the default 7B-parameter model in the example below, we recommend using a larger model for retrieval-augmented question-answering use cases like this in order to reduce hallucinations. You can use the default 13B-parameter model by supplying use_larger=True in the call to onprem.LLM.
llm.ingest("./sample_data/")2023-09-05 14:47:43.265584: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Loading new documents: 100%|██████████████████████| 2/2 [00:00<00:00, 16.59it/s]Creating new vectorstore
Loading documents from ./sample_data/
Loaded 11 new documents from ./sample_data/
Split into 62 chunks of text (max. 500 chars each)
Creating embeddings. May take some minutes...
Ingestion complete! You can now query your documents using the LLM.ask methodresult = llm.ask("What is ktrain? Remember to only use the provided context.") KTrain is a low-code library for augmented machine learning that automates or semi-automates various aspects of the ML workow, from data preprocessing to model training and application. It is inspired by other low-code and no-code open-source ML libraries such as fastai and ludwig, and is intended to help democratize machine learning by enabling beginners and domain experts with minimal programming or data science expertise.Pro-Tip: If you see the model hallucinating answers, you can supply use_larger=True to onprem.LLM and use the slightly larger default model better-suited to this use case (or supply the URL to a different model of your choosing to LLM), which can provide better performance.