cmc-csci143

cmc-csci143 copied to clipboard

Hi,

I have copied all the files from CS40 twitter clone to my project directory in services/web. When I run:

docker-compose up -d --build

and open localhost:3214, I get this error:

lambda-server:~/bigdata/final_project $ channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused

Everything was working fine until I added sql queries to my project.py file to interact with the twitter.db database. I think the issue has to do with problems connecting to twitter.db in my project directory, and I have been working on fixing the docker-compose.yml file to fix the issue.

This is my docker-compose.yml file:

lambda-server:~/bigdata/final_project $ cat docker-compose.yml
version: '3.8'

services:
  web:
    build: ./services/web
    command: python manage.py run -h 0.0.0.0
    volumes:
      - ./services/web/:/usr/src/app/
    ports:
        - 3214:3214
    env_file:
      - ./.env.dev
    depends_on:
      - db
      - postgres

  postgres:
    build: ./services/postgres
    command: postgres -c config_file=/etc/postgresql.conf
    volumes:
      - ./:/tmp/db
      - postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=pass
      - PGUSER=postgres
    ports:
      - 9373:5432

  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=hello_flask
      - POSTGRES_PASSWORD=hello_flask
      - POSTGRES_DB=hello_flask_dev

volumes:
  postgres_data:
  postgres:

This is my services/web directory:

lambda-server:~/bigdata/final_project $ ls services/web
Dockerfile       entrypoint.prod.sh  manage.py  requirements.txt
Dockerfile.prod  entrypoint.sh       project

This is my services/web/project directory:

lambda-server:~/bigdata/final_project $ ls services/web/project/
config.py     __init__.py            project.py   templates
db_access.py  markdown_compiler2.py  __pycache__  twitter_clone.db
db_create.py  media                  static

The error message

channel 4: open failed: connect failed: Connection refused

is a result of port forwarding not working. This happens when the ssh client connects to a port that does not have anything running (and so the connection is refused).

My guess is that you either:

1. used the wrong port in you ssh command, or
2. if you used the right port, then your docker-compose.yml file is incorrect. Specifically, the ports line of the web service has 3214:3214 listed. But the default port for flask (without nginx) is 5000. So 3214:5000 is what I would expect to see.

I get the same error:

lambda-server:~/bigdata/final_project $ channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused

I used this ssh:

amamnoon@abizer:~$ ssh [email protected] -p 5055 -L localhost:3214:localhost:5000

I updated my docker-compose.yml file:

lambda-server:~/bigdata/final_project $ cat docker-compose.yml
version: '3.8'

services:
  web:
    build: ./services/web
    command: python manage.py run -h 0.0.0.0
    volumes:
      - ./services/web/:/usr/src/app/
    ports:
        - 3214:5000
    env_file:
      - ./.env.dev
    depends_on:
      - db
      - postgres

  postgres:
    build: ./services/postgres
    volumes:
      - ./:/tmp/db
      - postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=pass
      - PGUSER=postgres
    ports:
      - 9373:5432

  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=hello_flask
      - POSTGRES_PASSWORD=hello_flask
      - POSTGRES_DB=hello_flask_dev

volumes:
  postgres_data:
  postgres:

I also updated my .env.dev file:

lambda-server:~/bigdata/final_project $ cat .env.dev
FLASK_APP=project/project.py
FLASK_DEBUG=1
FLASK_RUN_PORT=5000
DATABASE_URL=postgresql://hello_flask:hello_flask@db:5432/hello_flask_dev
SQL_HOST=db
SQL_PORT=5432
DATABASE=postgres
APP_FOLDER=/usr/src/app

Should I changed my manage.py file somehow to run the db_create.py?

lambda-server:~/bigdata/final_project $ cat services/web/manage.py
from flask.cli import FlaskGroup

from project import app, db, User
from project.db_create import create_database_and_populate

cli = FlaskGroup(app)


@cli.command("create_db")
def create_db():
    db.drop_all()
    db.create_all()
    db.session.commit()

@cli.command("seed_db")
def seed_db():
    db.session.add(User(email="[email protected]"))
    db.session.commit()

@cli.command("initialize_database")
def initialize_database():
    create_database_and_populate()

if __name__ == "__main__":
    cli()

I chose not to change it because I manually ran:

python3 db_create.py

This created the twitter_clone.db in my project directory

Hi @abizermamnoon did you figure this out? Is this still what your docker-compose and .env files look like for the dev environment?

Hi, Yes, I figured it out. Thanks to @henrylong612 and @tylerheadley This is my .env.dev file:

lambda-server:~/final_proj1 (master *) $ cat .env.dev
FLASK_APP=project/__init__.py
FLASK_DEBUG=1
FLASK_RUN_PORT=3214
DATABASE_URL=postgresql://postgres:pass@localhost:9373
SQL_HOST=postgres
SQL_PORT=5432
DATABASE=postgres
APP_FOLDER=/usr/src/app

This is my docker-compose.yml file:

lambda-server:~/final_proj1 (master *) $ cat docker-compose.yml
version: '3.8'

services:
  web:
    build: ./services/web
    command: python manage.py run -h 0.0.0.0
    volumes:
      - ./services/web/:/usr/src/app/
    ports:
        - 3214:3214
    env_file:
      - ./.env.dev
    depends_on:
      - postgres
  postgres:
    build: services/postgres
    volumes:
      - ./:/tmp/db
      - postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=pass
      - PGUSER=postgres
    ports:
      - 9373:5432

volumes:
  postgres:

I also used the following code in my init.py file:

engine = sqlalchemy.create_engine("postgresql://postgres:pass@postgres:5432", connect_args={
    'application_name': '__init__.py',
    })
connection = engine.connect()

thank you @abizermamnoon! i have a couple questions if you're able..

the 3214:3214 in your web ports, that works because of FLASK_RUN_PORT=3214? or is that separate? i was under the impression our web service should be $port we're mapping to something on our computer:5000

i've also been setting POSTGRES_DB=hello_flask_dev in the environment section of my dev database? is that not necessary? are you, like, using the same database for the dev and the prod environment? or something like that?

apologies if this stuff is answered in the videos/CS40 i haven't gotten there yet