from onprem import LLM
import os
llm = LLM(model_url='https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf',
prompt_template= "<s>[INST] {prompt} [/INST]",
n_gpu_layers=33,
temperature=0,
verbose=True)Use Prompts to Solve Problems
This notebook shows various examples of using OnPrem.LLM to solve different tasks.
Setup the LLM instance
In this notebook, we will use a model called Mistral-7B-Instruct-v0.2. When selecting a model, it is important to inspect the model’s home page and identify the correct prompt format. The prompt format for this model is located here, and we will supply it directly to the LLM constructor along with the URL to the specific model file we want (i.e., mistral-7b-instruct-v0.2.Q4_K_M.gguf). We will offload layers to our GPU(s) to speed up inference using the n_gpu_layers parameter. (For more information on GPU acceleration, see here.) For the purposes of this notebook, we also supply temperature=0 so that there is no variability in outputs. You can increase this value for more creativity in the outputs.
Information Extraction
This is an example of zero-shot prompting:
prompt = """Extract the names of people in the supplied sentences. Separate the names with commas.
[Sentence]: I like Cillian Murphy's acting. Florence Pugh is great, too.
[People]:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) Cillian Murphy, Florence PughA more complicated example of Information Extraction using few-shot prompting:
prompt = """ Extract the Name, Current Position, and Current Company from each piece of Text.
Text: Alan F. Estevez serves as the Under Secretary of Commerce for Industry and Security. As Under Secretary, Mr. Estevez leads
the Bureau of Industry and Security, which advances U.S. national security, foreign policy, and economic objectives by ensuring an
effective export control and treaty compliance system and promoting U.S. strategic technology leadership.
A: Name: Alan F. Estevez | Current Position: Under Secretary | Current Company: Bureau of Industry and Security
Text: Pichai Sundararajan (born June 10, 1972[3][4][5]), better known as Sundar Pichai (/ˈsʊndɑːr pɪˈtʃaɪ/), is an Indian-born American
business executive.[6][7] He is the chief executive officer (CEO) of Alphabet Inc. and its subsidiary Google.[8]
A: Name: Sundar Pichai | Current Position: CEO | Current Company: Google
Text: Norton Allan Schwartz (born December 14, 1951)[1] is a retired United States Air Force general[2] who served as the 19th Chief of Staff of the
Air Force from August 12, 2008, until his retirement in 2012.[3] He previously served as commander, United States Transportation Command from
September 2005 to August 2008. He is currently the president of the Institute for Defense Analyses, serving since January 2, 2020.[4]
A:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) Name: Norton Allan Schwartz | Current Position: President | Current Company: Institute for Defense AnalysesGrammar Correction
prompt = """Correct the grammar and spelling in the supplied sentences. Here are some examples.
[Sentence]:
I love goin to the beach.
[Correction]: I love going to the beach.
[Sentence]:
Let me hav it!
[Correction]: Let me have it!
[Sentence]:
It have too many drawbacks.
[Correction]: It has too many drawbacks.
[Sentence]:
I do not wan to go
[Correction]:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) I do not want to go.Classification
prompt = """Classify each sentence as either positive, negative, or neutral. Here are some examples.
[Sentence]: I love going to the beach.
[[Classification]: Positive
[Sentence]: It is 10am right now.
[Classification]: Neutral
[Sentence]: I just got fired from my job.
[Classification]: Negative
[Sentence]: The reactivity of your team has been amazing, thanks!
[Classification]:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) Positive.Paraphrasing
prompt = """Paraphrase the following text delimited by triple backticks using a single sentence.
```After a war lasting 20 years, following the decision taken first by President Trump and then by President Biden to withdraw American troops, Kabul, the capital of Afghanistan, fell within a few hours to the Taliban, without resistance.```
"""
saved_output = llm.prompt(prompt) In the aftermath of a protracted 20-year war, following the respective decisions made by Presidents Trump and Biden to withdraw American troops, Kabul, the Afghan capital, succumbed to the Taliban within a few hours, without any significant resistance.Few-Shot Answer Extraction
prompt = """ Compelte the correct answer based on the Context. Answer should be a short word or phrase from Context.
[Question]: When was NLP Cloud founded?
[Context]: NLP Cloud was founded in 2021 when the team realized there was no easy way to reliably leverage Natural Language Processing in production.
[Answer]: 2021
[Question]: What did NLP Cloud develop?
[Context]: NLP Cloud developed their API by mid-2020 and they added many pre-trained open-source models since then.
[Answer]: API
[Question]: When can plans be stopped?
[Context]: All plans can be stopped anytime. You only pay for the time you used the service. In case of a downgrade, you will get a discount on your next invoice.
[Answer]: Anytime
[Question]: Which plan is recommended for GPT-J?
[Context]: The main challenge with GPT-J is memory consumption. Using a GPU plan is recommended.
[Answer]:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) GPU planThe next three sections are on 1) generating product descriptions, 2) tweet generation, and 3) email draft generation. While Mistral-7B performs well on such text generation tasks, Zephyr-7B may perform better.
Generating Product Descriptions
prompt = """Generate a short Sentence from the Keywords. Here are some examples.
[Keywords]: shoes, women, $59
[Sentence]: Beautiful shoes for women at the price of $59.
[Keywords]: trousers, men, $69
[Sentence]: Modern trousers for men, for $69 only.
[Keywords]: gloves, winter, $19
[Sentence]: Amazingly hot gloves for cold winters, at $19.
[Keywords]: t-shirt, men, $39
[Sentence]:"""
saved_output = llm.prompt(prompt, stop=['\n\n']) Stylish men's t-shirts available for $39.Tweet Generation
prompt = """Generate a tweet based on the supplied Keyword. Here are some examples.
[Keyword]:
markets
[Tweet]:
Take feedback from nature and markets, not from people
###
[Keyword]:
children
[Tweet]:
Maybe we die so we can come back as children.
###
[Keyword]:
startups
[Tweet]:
Startups should not worry about how to put out fires, they should worry about how to start them.
###
[Keyword]:
climate change
[Tweet]:"""
saved_output = llm.prompt(prompt) Climate change is not a problem for our grandchildren to solve, it's a problem for us to solve for our grandchildren. #actonclimate #climateactionGenerating an Email Draft
prompt = """Generate an email introducing Tesla to shareholders."""
saved_output = llm.prompt(prompt) Subject: Exciting New Developments at Tesla - Shareholder Update
Dear Valued Tesla Shareholder,
We are thrilled to update you on the latest developments and innovations at Tesla.
Firstly, we are proud to announce that our Model Y SUV has officially gone into production at our new factory in Berlin, Germany. This marks an important milestone for Tesla as we continue to expand our vehicle lineup and global manufacturing footprint.
In addition to the Model Y production ramp-up in Europe, we are also making significant progress on several other fronts. For instance, we recently unveiled the latest version of our Autopilot advanced driver assistance system (ADAS). This new iteration of Autopilot features significant improvements in object detection and classification, as well as enhanced lane centering capabilities. These advancements are designed to help make Autopilot even safer and more effective for our customers.
Another exciting development at Tesla relates to our ongoing efforts to accelerate the transition to sustainable energy. In this regard, we are pleased to announce that we have recently signed a major new agreement with the utility company NextEra Energy Resources, LLC (NextEra) to develop and construct up to 2 gigawatts (GW) of new solar projects in the United States. This represents one of the largest renewable energy partnerships in history, and it is an important step forward in our mission to help lead the world's transition to sustainable energy.
In conclusion, we are extremely proud of the progress that Tesla has made over the past year, and we are excited about the many new developments and innovations that are on the horizon for our company. We would like to express our deepest gratitude to all of our dedicated employees, customers, investors, and partners who have supported us along the way. Together, we are helping to shape a brighter and more sustainable future for ourselves, our planet, and generations to come.
Thank you for your continued support and trust in Tesla. We look forward to keeping you updated on all of the exciting new developments and innovations that are happening at Tesla.
Warm regards,
Elon Musk
CEO, Tesla, Inc.Talk to Your Documents
llm.ingest("./sample_data/")Creating new vectorstore at /home/amaiya/onprem_data/vectordb
Loading documents from ./sample_data/
Loaded 12 new documents from ./sample_data/
Split into 153 chunks of text (max. 500 chars each)
Creating embeddings. May take some minutes...
Ingestion complete! You can now query your documents using the LLM.ask or LLM.chat methodsLoading new documents: 100%|██████████████████████| 3/3 [00:00<00:00, 22.00it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00, 2.78s/it]result = llm.ask("What is ktrain?") Ktrain is a low-code machine learning library that facilitates the full machine learning workflow from curating and preprocessing inputs to training, tuning, troubleshooting, and applying models. It is well-suited for domain experts who may have less experience with machine learning and software coding.Pro-Tip: You can try different models or re-phrase the question, which may provide better performance for certain tasks. For instance, you can use the default Wizard-Vicuna-7B-Uncensored model, which can also solve the above tasks, as shown in this example Google Colab notebook of OnPrem.LLM.