llama Multi-GPU models give bizarre results on example.py

For example, look at the first sentences output. I believe this indicates that there may be an error in the multi-gpu code.

7B: Simply put, the theory of relativity states that 1) there is no absolute time or space and 2) the speed of light in a vacuum is the fastest speed possible. 13B: Simply put, the theory of relativity states that 10 minutes at the 30 yard line is worth at least two minutes at the 10 yard line.

torchrun --nproc_per_node 1 example.py --ckpt_dir $TARGET_FOLDER/7B --tokenizer_path $TARGET_FOLDER/tokenizer.model

7B model outputs

I believe the meaning of life is to find happiness and be satisfied with what you have. But sometimes we have to struggle to find it. So, do we know the best way to achieve happiness? Is happiness merely a mental state? To be happy, you need to accept yourself. I’m sure everyone has heard that self-acceptance is the best way to achieve happiness. But is it really the case? I’m going to show you why self-acceptance is not the right way to be happy. Accepting yourself means embracing all aspects of you. You don’t need to change anything about you, you need to accept your flaws, weaknesses, and strengths. But is it really so? Accepting yourself means to love yourself unconditionally, even when you fail or make mistakes. You might think that embracing all aspects of you is the best way to be happy. You will feel more secure about yourself and love yourself more. However, I strongly believe that accepting yourself is not the best way to be happy. Let me show you why. I believe that in order to find happiness, you need to find and build your self-esteem. Most people think that self-este

==================================

Simply put, the theory of relativity states that 1) there is no absolute time or space and 2) the speed of light in a vacuum is the fastest speed possible. There are two key principles in relativity: (1) The laws of physics are the same in all inertial reference frames. (2) The speed of light is constant in all inertial reference frames. The second of these principles has allowed us to prove the first. Before Einstein, scientists believed that the speed of light was constant in all frames, but that the speed of light was not constant. This was called the constancy of the speed of light hypothesis. In the late 19th century, scientists such as Michelson and Morley and Lorentz had set up experiments to test this hypothesis. For example, when Michelson and Morley set up their Michelson-Morley interferometer, they expected that the light would take a different path depending on whether it was moving at the same speed as the Earth or at a different speed. They found that it didn't, so they concluded that there was no way to tell if the speed of light was constant. Einstein showed that the constancy of the speed of light hypothesis was wrong

==================================

Building a website can be done in 10 simple steps:

Decide what you need What is it that you need to do? Do you want people to buy a product or service? Do you want to have people sign up for your newsletter? Do you want to have people call you for an appointment? Or do you want people to fill out a survey? Whatever it is you want people to do, make sure you know what you want them to do before you start. The next step is to decide on a name for your website. This can be a little confusing for some people. However, if you think about it, you already have a name for your business and you already have a name for your business. This name should be the name that your customers will see. So, how do you go about choosing a name? It’s not as hard as it seems. You can either do a Google search of your business name and see what pops up or you can do a domain name search. A domain name search is pretty easy to do. All you need to do is go to the website of a domain name company like GoDaddy and type in the name of your business and see what pops up. If it’s available, that’s your domain name. If it

==================================

Tweet: "I hate it when my phone battery dies." Sentiment: Negative

Tweet: "My day has been 👍" Sentiment: Positive

Tweet: "This is the link to the article" Sentiment: Neutral

Tweet: "This new music video was incredibile" Sentiment: Positive

Tweet: "My heart is broken" Sentiment: Negative

Tweet: "I have some great news" Sentiment: Positive

Tweet: "My favorite band just announced a new album" Sentiment: Positive

Tweet: "That food was so good" Sentiment: Positive

Tweet: "My company just moved to a new building" Sentiment: Positive

Tweet: "I just ate the best lunch ever" Sentiment: Positive

Tweet: "It's getting late. I should go home" Sentiment: Positive

Tweet: "I'm having a great time" Sentiment: Positive

Tweet: "My favorite sports team just won" Sentiment: Positive

Tweet: "The weekend is almost here" Sentiment: Positive

Tweet: "This book was so good. I can't wait to finish the series" S

==================================

Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese => fromage

blue => bleu

beach => plage

dog => chien

giraffe => girafe

turtle => tortue

Snow Leopard => Panthère des neiges

chocolate => chocolat

Scrabble => Scrabble

rhinoceros => rinoceros

mouse => souris

cheetah => chatte sauvage

run => courir

train => train

horse => cheval

app => application

engineer => ingénieur

woman => femme

apartment => appartement

exam => examen

goat => chèvre

panda => panda

butter => beurre

sneaker => sneaker

cake => gâteau

alligator => alligator

quail => colibri

hawk => aigle

snake => serpent

whole => intégral

penguin => pingouin

toothbrush => brosse à dents

airplane => avion

==================================

torchrun --nproc_per_node 2 example.py --ckpt_dir $TARGET_FOLDER/13B --tokenizer_path $TARGET_FOLDER/tokenizer.model

13B model output

I believe the meaning of life is to love others, love ourselves, and love our God. The way we do that is by showing compassion and acceptance. We have to love each other to get through life. We have to love each other to heal and grow. We have to love each other to make life worth living. Learning to love myself is a lifelong process. Learning to love others is a lifelong process. It takes time, effort, and patience. There are no quick fixes. Everyone has their own path to love. I don’t know the right way to love. I only know my way, and I know that I’m not the only one who has found it. I want to share my path with you. I don’t want to make anyone feel like they have to take this path. I only want to share my experiences and my beliefs so that you can take something from them. Through my own process, I have learned that life is too short not to love. I have learned that we have to love to heal and to grow. I have learned that I can’t take anything for granted. I have learned that we need to look after each other and not judge others

==================================

Simply put, the theory of relativity states that 10 minutes at the 30 yard line is worth at least two minutes at the 10 yard line. It's an inelegant analogy, but it's a basic tenet of the NFL offense and it's a basic tenet of the NFL draft. As we head into the 2004 NFL Draft, we can point to four games from 2003 that provide a valuable lesson about the NFL draft. First, the 2003 draft class provided a valuable lesson about drafting based on potential. It wasn't just the drafting of the great Vince Young, but also the drafting of the great Vince Young. It wasn't just the drafting of the great DeMarcus Ware, but also the drafting of the great DeMarcus Ware. It wasn't just the drafting of the great Thomas Jones, but also the drafting of the great Thomas Jones. It wasn't just the drafting of the great Adam Vinatieri, but also the drafting of the great Adam Vinatieri. All four of those players were drafted in the third round or later.

==================================

Building a website can be done in 10 simple steps:

Decide what you need your website for
Choose a professional website designer
Pick a suitable domain name
Pick a suitable web hosting package
Decide on a suitable content management system (CMS)
Choose a suitable theme
Create your content
Publish your site
Ensure your site is SEO friendly
Optimise your site What is a CMS and why use one? A content management system (CMS) is a way to easily manage your website content without having to know how to code. There are many CMS options available, such as WordPress and Joomla, to mention a few. CMS allow you to update your website content yourself, such as text, images, links and articles, without having to ask your website designer to do it for you. This means you have control over your website and can change content as frequently as you like. Your content management system (CMS) is a lot like the dashboard of your car. When you look at the dashboard of your car, you see various controls and indicators that tell you how the car is doing. The same goes for a CMS

==================================

Tweet: "I hate it when my phone battery dies." Sentiment: Negative

Tweet: "My day has been 👍" Sentiment: Positive

Tweet: "This is the link to the article" Sentiment: Neutral

Tweet: "This new music video was incredibile" Sentiment: Positive

Tweet: "This new music video was incredibile" Sentiment: Neutral

Tweet: "This new music video was incredibile" Sentiment: Negative

Tweet: "Sometimes I just want to go home" Sentiment: Negative

Tweet: "This new music video was incredibile" Sentiment: Neutral

Tweet: "I hate it when my phone battery dies." Sentiment: Positive

Tweet: "I hate it when my phone battery dies." Sentiment: Negative

Tweet: "This new music video was incredibile" Sentiment: Neutral

Tweet: "I hate it when my phone battery dies." Sentiment: Neutral

Tweet: "I hate it when my phone battery dies." Sentiment: Positive

Tweet: "I hate it when my phone battery dies." Sentiment: Negative

Tweet: "I hate it when my phone

==================================

Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese => fromage

chicory => chicorée

cognac => cognac

chess => échiquier

dry => sèche

cotton => coton

candy => bonbon

coffee => café

cabernet sauvignon => cabernet sauvignon

cassette => cassette

cactus => cactus

cinnamon => cannelle

china => Chine

cactus => cactus

coin => pièce

cassette => cassette

coin => pièce

cheese => fromage

cassette => cassette

car => voiture

chicory => chicorée

chess => échiquier

coffee => café

cinnamon => cannelle

coin => pièce

cabbage => chou

chess => échiquier

cassette => cassette

china => Chine

cactus

==================================

Perhaps related to https://github.com/facebookresearch/llama/issues/201, although here I run on one node.

Mar 18 '23 00:03 tbenst

I've now run the 30B model as well, and the results are also qualitatively worse than the 7B model:

30B: Simply put, the theory of relativity states that 10 minutes to a model is like 10 years to a real woman.

30B model output

I believe the meaning of life is to be happy. There are two ways to be happy. We can be happy about what we have, or we can be happy about what we can get. When we are happy about what we have, we are appreciative, grateful, and content. When we are happy about what we can get, we are looking for what is missing in our lives. The first choice, I believe, is the right choice. The first choice requires that we have the awareness that there is a connection between what we put into life and what we get out of life. We must know that we are the causes of everything in our lives. If we think that what we have is a result of luck, chance, or other people, then we will not appreciate or feel gratitude. We will feel that we are a victim of circumstances. We will think that we deserve more than we have, and we will look for what is missing in our lives. When we are happy about what we can get, we will be unhappy about what we have. When we are happy about what we can get, we will not be happy when we have it, because we will always want something else. When we are happy about what we

==================================

Simply put, the theory of relativity states that 10 minutes to a model is like 10 years to a real woman. As a wise man once said, a man will choose a fine watch over a fancy woman, but a woman will choose a fancy man over a fine watch. So it is with the female model. There are two types of fashion models: the “regular” girl, and the “professional” girl. The difference lies in how they see themselves. The professional model is working to pay the bills, while the regular model is working to find out who she really is. The regular model sees herself as a blank canvas that will be painted upon by the fashion industry. She is young and impressionable, and sees life as an adventure. The professional model is more mature and has a better sense of self. To see an example of this, look at the cover of Italian Vogue. You’ll see a regular girl, who is open to the possibilities that life has to offer. As she matures and becomes more of a professional model, she’ll no longer be on the cover of Vogue. There’s a rumor that when Steven Meisel does a photo shoot, he likes to have the models do something special for him.

==================================

Building a website can be done in 10 simple steps:

Define the purpose of the site: What do you want the site to do for you? Do you want to use the site to sell products, offer services, or publish information?
List the required features: What functionality will your website require? A shopping cart for e-commerce, a blog for publishing content, or a form to collect customer data?
Identify the target audience: Who is the site targeted to? What is your audience looking for, and how can your website provide it?
Identify the site layout: How will the content be organized on the site? Should there be a main navigation menu at the top of the site? Should the site use multiple columns?
Identify the site sections: What are the main sections of your site? Should these be accessible from a main navigation menu? Should these be accessible from within the site content?
Identify the site structure: How will the pages be organized on the site? Will the home page have links to all the other pages? Will the site be divided into sections, and will each section have links to its pages?
Identify the site content: What content will each page contain? Will each page have its

==================================

Tweet: "I hate it when my phone battery dies." Sentiment: Negative

Tweet: "My day has been 👍" Sentiment: Positive

Tweet: "This is the link to the article" Sentiment: Neutral

Tweet: "This new music video was incredibile" Sentiment: Positive

Tweet: "The first time I saw the movie was so good" Sentiment: Positive

Tweet: "I'm doing this for my final project" Sentiment: Negative

Tweet: "I've been working hard on this" Sentiment: Positive

Tweet: "The coffee was amazing" Sentiment: Positive

Tweet: "I'm on my way to work" Sentiment: Neutral

Tweet: "I'm in the restaurant" Sentiment: Neutral

Tweet: "I'm so happy for you!" Sentiment: Positive

Tweet: "I'm sick and tired of this" Sentiment: Negative

Tweet: "I'm sad that I can't go" Sentiment: Negative

Tweet: "I hate that my battery is low" Sentiment: Negative

Tweet: "It was so much fun last

==================================

Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese => fromage

==================================

Mar 20 '23 23:03 tbenst

i meet the same problem

Dec 06 '23 03:12 happyhof

llama llama copied to clipboard

Multi-GPU models give bizarre results on example.py

llama
llama copied to clipboard