Create a Guide: Web Scraping

[akhaleghi]

Overview

This issue contains resources to help community members learn more about web scraping in Python, including the use of APIs.

Action Items

• [ ] Gather resources, including links to relevant online courses, and tutorial/how-to web content for inspiration or further reading for users.
• [ ] Create an outline of what you will cover.
  • include steps on how to get started for volunteers new to web scraping
• [ ] Create a draft template, either in Markdown format in this issue or a Google doc in the Data Science google drive
• [ ] Review the guide with Data Science Communities of Practice
• [ ] Present to Hack for LA leadership team for sign off
  • [ ] Once approved, remove the "Guide: Leadership Review" label and add the "Guide: Place Guide" label
• [ ] Include link to guide under Resources if you add it as a template in .github

Resources/Instructions

Hack for LA Web Scraping Tutorial with Selenium/Docker/Python

[niralishah8539]

@chinaexpert1 I went through the resource and instructions, but not able to figure it out. I am gonna need a little guidance on what exactly I have to do here.

[chinaexpert1]

@*niralishah8539 *I believe the task is to create a step by step tutorial how to scrape a web page and save the contents, or a bunch of websites programmatically. You probably will need an explanation, a script, and instructions for someone to follow. Does this help?

On Tue, Aug 26, 2025, 5:32 PM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971

@chinaexpert1 https://github.com/chinaexpert1 I went through the resource and instructions, but not able to figure it out. I am gonna need a little guidance on what exactly I have to do here.

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6SRH5XCEMOSN47XAVZ5T33PT4ADAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRWGI3TCOJXGE . You are receiving this because you were mentioned.Message ID: @.***>

[parcheesime]

I've created step by step tutorial. This was in 2022, and am now working on updating as api links/format can change, keys can expire, and code may need updating.

I read the tutorial document and it seems that it does not have code and will help get the code in there.

Happy to meet and show the code.

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Tue, Aug 26, 2025, 6:03 PM ANDREW W TAYLOR @.***> wrote:

chinaexpert1 left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3226330294 @*niralishah8539 *I believe the task is to create a step by step tutorial how to scrape a web page and save the contents, or a bunch of websites programmatically. You probably will need an explanation, a script, and instructions for someone to follow. Does this help?

On Tue, Aug 26, 2025, 5:32 PM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) < https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971>

@chinaexpert1 https://github.com/chinaexpert1 I went through the resource and instructions, but not able to figure it out. I am gonna need a little guidance on what exactly I have to do here.

— Reply to this email directly, view it on GitHub < https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AA6SRH5XCEMOSN47XAVZ5T33PT4ADAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRWGI3TCOJXGE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3226330294, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDAJOAMIQFKF2S6F3BHZSD3PT7VJAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRWGMZTAMRZGQ . You are receiving this because you were assigned.Message ID: @.***>

[chinaexpert1]

Ok @parcheesime thank you

@*niralishah8539 * You are now a reviewer on the tutorial. Start reading through it keeping track of anything you think should be added or subtracted.

On Wed, Aug 27, 2025, 7:15 AM Aletia Trepte @.***> wrote:

parcheesime left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3228392019 I've created step by step tutorial. This was in 2022, and am now working on updating as api links/format can change, keys can expire, and code may need updating.

I read the tutorial document and it seems that it does not have code and will help get the code in there.

Happy to meet and show the code.

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Tue, Aug 26, 2025, 6:03 PM ANDREW W TAYLOR @.***> wrote:

chinaexpert1 left a comment (hackforla/data-science#130) < https://github.com/hackforla/data-science/issues/130#issuecomment-3226330294>

@*niralishah8539 *I believe the task is to create a step by step tutorial how to scrape a web page and save the contents, or a bunch of websites programmatically. You probably will need an explanation, a script, and instructions for someone to follow. Does this help?

On Tue, Aug 26, 2025, 5:32 PM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) <

https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971>

@chinaexpert1 https://github.com/chinaexpert1 I went through the resource and instructions, but not able to figure it out. I am gonna need a little guidance on what exactly I have to do here.

— Reply to this email directly, view it on GitHub <

https://github.com/hackforla/data-science/issues/130#issuecomment-3226271971>,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AA6SRH5XCEMOSN47XAVZ5T33PT4ADAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRWGI3TCOJXGE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/hackforla/data-science/issues/130#issuecomment-3226330294>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AJDAJOAMIQFKF2S6F3BHZSD3PT7VJAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRWGMZTAMRZGQ>

. You are receiving this because you were assigned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3228392019, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6SRH2T35YTSRANYHGJ6OT3PW4QTAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMRYGM4TEMBRHE . You are receiving this because you were mentioned.Message ID: @.***>

[ExperimentsInHonesty]

Hi @niralishah8539

Please start with an outline of what you think should be on the tutorial page. Then show that outline to us at the team meeting and we can give you some feedback. I also updated the top part of this issue to make more sense.

Bonnie

chinaexpert1 said: @niralishah8539 I believe the task is to create a step by step tutorial how to scrape a web page and save the contents, or a bunch of websites programmatically. You probably will need an explanation, a script, and instructions for someone to follow. Does this help?

[niralishah8539]

Hi @ExperimentsInHonesty

Yes, I will!

Best, Nirali

[niralishah8539]

I've created step by step tutorial. This was in 2022, and am now working on updating as api links/format can change, keys can expire, and code may need updating.

I read the tutorial document and it seems that it does not have code and will help get the code in there.

Happy to meet and show the code.

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Tue, Aug 26, 2025, 6:03 PM ANDREW W TAYLOR @.***> wrote: …

Hi, Sorry for the delay in response. I really appreciate your help! Through my academic projects, I have learned about web scraping/crawling using Python, and I am planning to apply the same skills. But I would love to work with you on expired keys, updating code, and if the format can change, that is something I haven't learn.

[chinaexpert1]

I've created step by step tutorial. This was in 2022, and am now working on updating as api links/format can change, keys can expire, and code may need updating. I read the tutorial document and it seems that it does not have code and will help get the code in there. Happy to meet and show the code. website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime On Tue, Aug 26, 2025, 6:03 PM ANDREW W TAYLOR @.***> wrote: …

Hi, Sorry for the delay in response. I really appreciate your help! Through my academic projects, I have learned about web scraping/crawling using Python, and I am planning to apply the same skills. But I would love to work with you on expired keys, updating code, and if the format can change, that is something I haven't learn.

@niralishah8539 please leave a formal update with the saved reply (progress, blockers, ETA, etc) where you list your specific blockers (as well as what you need from me or Bonnie) and what are the next steps for this issue. Thank you

[ExperimentsInHonesty]

@niralishah8539 Please provide update

Instructions

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (if necessary): "Add any pictures that will help illustrate what you are working on."

You can use this template

1. Progress: 
2. Blockers: 
3. Availability:
4. ETA:
5. Pictures (if necessary): 

[niralishah8539]

Hi

I've created step by step tutorial. This was in 2022, and am now working on updating as api links/format can change, keys can expire, and code may need updating. I read the tutorial document and it seems that it does not have code and will help get the code in there. Happy to meet and show the code. website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime On Tue, Aug 26, 2025, 6:03 PM ANDREW W TAYLOR @.***> wrote: …

Hi, Sorry for the delay in response. I really appreciate your help! Through my academic projects, I have learned about web scraping/crawling using Python, and I am planning to apply the same skills. But I would love to work with you on expired keys, updating code, and if the format can change, that is something I haven't learn.

@niralishah8539 please leave a formal update with the saved reply (progress, blockers, ETA, etc) where you list your specific blockers (as well as what you need from me or Bonnie) and what are the next steps for this issue. Thank you

@chinaexpert1 Hi, I was gonna update in the meeting today as Bonnie said. But below is the things I am thinking to include and my next step to create a Jupyter Notebook step by step for web scraping.

• Introduction to Web Scraping
• Setting up the Environment
• Understanding HTML Structure
• Basic Scraping
• Storing Scraped Data
• Example project
• Common problems (Handling errors such as missing elements)

[niralishah8539]

Reopening the ticket. Closed by mistake.

[niralishah8539]

@chinaexpert1 @ExperimentsInHonesty

1. Progress: Made an outline
2. Blockers:
3. Availability: (None) This week has travel plans
4. ETA: 20-25 days
5. Pictures (if necessary):

[niralishah8539]

Made an Outline:

• Introduction to Web Scraping
• Setting up the Environment
• Understanding HTML Structure
• Basic Scraping
• Storing Scraped Data
• Example project
• Common problems (Handling errors such as missing elements)

[ExperimentsInHonesty]

Please try to use this checklist and provide feedback if anything in it does not make sense

Generic Markdown Template for Intermediate Tutorials

✅ Tutorial Creation Checklist

• [ ] Title & Overview: Clear, descriptive tutorial name + short overview of what it covers.
• [ ] Purpose: Why the tutorial matters, what problem it solves, or what skills it teaches.
• [ ] Prerequisites: List of assumed knowledge or tools students should already have.
• [ ] Setup Instructions: Step-by-step guide for installing or preparing required tools/software.
• [ ] Core Concepts: Break tutorial into sections with explanations and examples.
• [ ] Hands-On Lessons/Exercises: Provide links, files, or in-line instructions for practice.
• [ ] Best Practices & Tips: Highlight common pitfalls, debugging help, or efficiency tricks.
• [ ] Examples: Any projects that we have done that illustrate this. If we don't have any, you may wish to make a sample, but it may not be required.
• [ ] Additional Resources: Curated links, videos, and references for further study.
• [ ] Contributors: Credit authors and collaborators.
• [ ] Issues Referenced: Link back to GitHub issues (if tutorial originated from repo discussions).

Template view

[Insert Tutorial Title Here]

Overview

[Insert content here]

Purpose

[Insert content here]

Prerequisites

[Insert content here]

Setup Before Workshop

[Insert content here]

Core Concepts

Concept 1: [Insert topic here]

[Insert content here]

Concept 2: [Insert topic here]

[Insert content here]

Concept 3: [Insert topic here]

[Insert content here]

Hands-On Lessons & Exercises

• [Insert lesson link or description here]
• [Insert exercise instructions here]

Best Practices & Tips

[Insert content here]

Examples

[Insert content here]

Additional Resources

[Insert content here]

Contributors

[Insert content here]

Issues Referenced

[Insert content here]

Template copy

## [Insert Tutorial Title Here]

## Overview
[Insert content here]

## Purpose
[Insert content here]

## Prerequisites
[Insert content here]

## Setup Before Workshop
[Insert content here]

## Core Concepts
### Concept 1: [Insert topic here]
[Insert content here]

### Concept 2: [Insert topic here]
[Insert content here]

### Concept 3: [Insert topic here]
[Insert content here]

## Hands-On Lessons & Exercises
- [Insert lesson link or description here]
- [Insert exercise instructions here]

## Best Practices & Tips
[Insert content here]

## Examples
[Insert content here]

## Additional Resources
[Insert content here]

## Contributors
[Insert content here]

## Issues Referenced
[Insert content here]

[niralishah8539]

Hi @chinaexpert1 @ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

[parcheesime]

Hi @chinaexpert1 @ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

Hi @niralishah8539 I am working on a web scraper tutorial. I would be happy to collaborate with you, so yes, I would love to see it. I am usually available on Thursdays, 4:30 PM PST or later. Would you like to meet?

[niralishah8539]

Hi @chinaexpert1 @ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

Hi @niralishah8539 I am working on a web scraper tutorial. I would be happy to collaborate with you, so yes, I would love to see it. I am usually available on Thursdays, 4:30 PM PST or later. Would you like to meet?

Hi @parcheesime, I would definitely like to meet, but I have just started a job and my work schedule is from mid-afternoon till 9 pm PST. And I do get off on Monday & Friday, would any of these days work for you?

[parcheesime]

Hi @chinaexpert1 @ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

Hi @niralishah8539 I am working on a web scraper tutorial. I would be happy to collaborate with you, so yes, I would love to see it. I am usually available on Thursdays, 4:30 PM PST or later. Would you like to meet?

Hi @parcheesime, I would definitely like to meet, but I have just started a job and my work schedule is from mid-afternoon till 9 pm PST. And I do get off on Monday & Friday, would any of these days work for you?

I can meet Friday in the morning/early afternoon, anywhere between 11 am and 1 pm, would somewhere in that time frame work? @niralishah8539

[niralishah8539]

Hi @chinaexpert1 @ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

Hi @niralishah8539 I am working on a web scraper tutorial. I would be happy to collaborate with you, so yes, I would love to see it. I am usually available on Thursdays, 4:30 PM PST or later. Would you like to meet?

Hi @parcheesime, I would definitely like to meet, but I have just started a job and my work schedule is from mid-afternoon till 9 pm PST. And I do get off on Monday & Friday, would any of these days work for you?

I can meet Friday in the morning/early afternoon, anywhere between 11 am and 1 pm, would somewhere in that time frame work? @niralishah8539

Sure, let's meet at 11:30 am on Friday.

[parcheesime]

Hey @niralishah8539, what is you hack4la slack handle?

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Mon, Sep 29, 2025 at 4:30 PM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3349383771

Hi @chinaexpert1 https://github.com/chinaexpert1 @ExperimentsInHonesty https://github.com/ExperimentsInHonesty I have made tutorial. Will you let me know what do you think? Where should I share it?

Hi @niralishah8539 https://github.com/niralishah8539 I am working on a web scraper tutorial. I would be happy to collaborate with you, so yes, I would love to see it. I am usually available on Thursdays, 4:30 PM PST or later. Would you like to meet?

Hi @parcheesime https://github.com/parcheesime, I would definitely like to meet, but I have just started a job and my work schedule is from mid-afternoon till 9 pm PST. And I do get off on Monday & Friday, would any of these days work for you?

I can meet Friday in the morning/early afternoon, anywhere between 11 am and 1 pm, would somewhere in that time frame work? @niralishah8539 https://github.com/niralishah8539

Sure, let's meet at 11:30 am on Friday.

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3349383771, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDAJOHK5B7J36XHJIR72WL3VG6LHAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNBZGM4DGNZXGE . You are receiving this because you were mentioned.Message ID: @.***>

[niralishah8539]

Hey @niralishah8539, what is you hack4la slack handle?

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime …

My Slack handle is Nirali Shah

[niralishah8539]

Hey @niralishah8539, what is you hack4la slack handle?

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime …

Hey, are we meeting tomorrow at 11:30 PDT?

[parcheesime]

I can meet at 1130 today!

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Thu, Oct 2, 2025, 11:30 AM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3362471747

Hey @niralishah8539 https://github.com/niralishah8539, what is you hack4la slack handle?

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime … <#m_502884725829844230_>

Hey, are we meeting tomorrow at 11:30 PDT?

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3362471747, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDAJOG4GVOPIOTXYWMH42T3VVVLHAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNRSGQ3TCNZUG4 . You are receiving this because you were mentioned.Message ID: @.***>

[niralishah8539]

I can meet at 1130 today!

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime … That's great! Please give me your email address so I can invite you to meeting.

[parcheesime]

Please message me on Slack, search for aletia. Thanks.

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime

On Fri, Oct 3, 2025, 10:50 AM niralishah8539 @.***> wrote:

niralishah8539 left a comment (hackforla/data-science#130) https://github.com/hackforla/data-science/issues/130#issuecomment-3366665926

I can meet at 1130 today!

website: portfolio https://parcheesime.github.io/portfolio-site/ https://sites.google.com/view/aletiatrepte/home linkedin: aletia-trepte https://www.linkedin.com/in/aletia-trepte/ github: parcheesime https://github.com/parcheesime … <#m_7938861308868738998_> That's great! Please give me your email address so I can invite you to meeting.

— Reply to this email directly, view it on GitHub https://github.com/hackforla/data-science/issues/130#issuecomment-3366665926, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDAJOBX2HYXBKC3GI4WKJ33V2ZOBAVCNFSM6AAAAACEG24GYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNRWGY3DKOJSGY . You are receiving this because you were mentioned.Message ID: @.***>

[chinaexpert1]

Please provide update @parcheesime

Instructions

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (if necessary): "Add any pictures that will help illustrate what you are working on."

You can use this template

1. Progress: 
2. Blockers: 
3. Availability:
4. ETA:
5. Pictures (if necessary): 

[parcheesime]

1. Progress:

Updated the README.md for the Ethical Web Scraping tutorial to include four lessons, including advanced scraping.

Refreshed scraper files to reflect current, active sites for demonstrations.

Met with another team member last month to review and align with their scraping documentation and align with Data Science Wiki Tutorial page.

2. Blockers:

Need repository access or confirmation to publish my repo to the Data Science Community GitHub org. Link to my repo, data scraping and collection

Scheduling and Selenium examples still need to be refined and tested.

3. Availability:

Available this week and next week for continued updates and repo integration.

4. ETA:

Plan to finish Selenium setup, scheduling section, and publish the repo in a few weeks.

[chinaexpert1]

Please provide update @parcheesime

Instructions

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (if necessary): "Add any pictures that will help illustrate what you are working on."

You can use this template

1. Progress: 
2. Blockers: 
3. Availability:
4. ETA:
5. Pictures (if necessary):