Building a Simple Proxy Server with Raspberry Pi

Discover the simplicity of setting up a proxy server on your Raspberry Pi using the user-friendly and open-source software known as Squid. In this tutorial, we provide a step-by-step guide, demonstrating its application for web scraping. However, the advantages of establishing a proxy server extend beyond this, encompassing enhanced security, efficient caching, accelerated networking requests, and streamlined connection management. Unlock the potential of your Raspberry Pi with this comprehensive tutorial on Squid proxy server setup!
Before reading the remainder, be sure to subscribe and support the channel if you have not!
Subscribe:
Support:
Hire me at UpWork to build your IoT projects:

Part 1: Setting up the Raspberry Pi

Initial Setup:
- Ensure your Raspberry Pi 4 is set up with Raspbian (or another compatible OS) and that it's connected to the internet.
- Access your Raspberry Pi terminal through SSH or directly using a monitor and keyboard.
Update and Upgrade Packages:
- Run sudo apt-get update and sudo apt-get upgrade to ensure all packages are up to date.
Install Squid:
- Execute sudo apt-get install squid.
- Once installed, the Squid service should start automatically.
Configure Squid:
- Backup the original configuration file: sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.backup.
- Edit the configuration file: sudo nano /etc/squid/squid.conf.
You can see the lines are uncommented:
acl localnet src 192.168.1.0/24
http_access allow localnet
This is needed to only allow devices on your local home network to connect to the proxy. It is a very simple setting; you can play with the config in this file to filter what IPs and what network security parameters you would like to configure. For the sake of this video, we keep it simple.
Restart Squid:
- Restart the Squid service to apply the changes: sudo systemctl restart squid.
Verify Squid is Running:
- Check the status of Squid: sudo systemctl status squid.
Get Raspberry Pi IP Address:
To get the Raspberry Pi IP address you can type in the command ifconfig in the terminal and look at the inet address. You will need this address to use on the Python script on your local computer that will send the scraping request to the Pi's IP address first as proxy!

Part 2: Local Computer Code

Create a Python Script on your local computer and run the following script, make sure you substitute your Pi IP address.
This script is designed to scrape web content from a specified URL (in this case, a search page on eBay for laptops) using the Python requests library for making HTTP requests and the BeautifulSoup library from bs4 for parsing HTML content. The script uses a proxy server to make the request. 
    In a real-world scenario, you'd likely want to add more functionality after parsing the HTML to extract and process the specific data you're interested in.

    Conclusion:

    Hope you enjoyed the quick tutorial, if the Python script ran it means that the requests are going through your Pi's IP address which can help you webscrape, let me know if you have any questions.
    If you enjoy the video, please subscribe to my channel Shilleh on Youtube in the video above, your support would be appreciated. Ping me for any questions, thanks everyone!

    Create a free account to access full content.

    All access to code and resources on ShillehTek.

    Signup Now

    Already a member? Sign In

    Explore More on Our Blog

    Interfacing the MPU6050 with Raspberry Pi Pico W in C++

    Interfacing the MPU6050 with Raspberry Pi Pico W in C++

    Interface with the MPU6050 using the Raspberry Pi Pico W in C++.

    How to Write your First C++ Program on the Raspberry Pi Pico W

    How to Write your First C++ Program on the Raspberry Pi Pico W

    Write your first C++ Program on the Pico W in a few simple steps.

    How to Use ThingSpeak with the Raspberry Pi Pico W

    How to Use ThingSpeak with the Raspberry Pi Pico W

    Learn how to create a real-time environmental monitoring system with the Raspberry Pi Pico W and ThingSpeak!

    How to Use ADS1115 with the Raspberry Pi (Part 1)

    How to Use ADS1115 with the Raspberry Pi (Part 1)

    Discover how to expand your Raspberry Pi projects by integrating the ADS1115 ADC for precise analog signal reading....

    How to Install Pip Packages in AWS Lambda Using Docker and ECR

    How to Install Pip Packages in AWS Lambda Using Docker and ECR

    Learn how to streamline AWS Lambda deployments by using Docker and Amazon Elastic Container Registry (ECR) to package...

    Create Tabular Product Descriptions on Your Shopify Store

    Create Tabular Product Descriptions on Your Shopify Store

    Enhance your Shopify store's product pages with our comprehensive guide on implementing tabular descriptions. Learn how to add a...

    SSH Into Raspberry Pi with Tailscale VPN

    SSH Into Raspberry Pi with Tailscale VPN

    Effortlessly access and manage your Raspberry Pi from anywhere using Tailscale's secure mesh VPN.

    Send Email with Lua and the ESP32

    Send Email with Lua and the ESP32

    In this tutorial, we delve into sending emails with the ESP32-S3 using Lua, focusing on the Xedge IDE's built-in SMTP...

    How to Code with Lua on ESP32 with XEdge32

    How to Code with Lua on ESP32 with XEdge32

    Learn how to set up Xedge32 and start coding on the ESP32-S3 with Lua programming!

    Stream Audio From Raspberry Pi to Local Computer

    Stream Audio From Raspberry Pi to Local Computer

    Discover the simplicity of streaming live audio directly from a USB microphone connected to your Raspberry Pi to...

    Back to blog

    Leave a comment

    Please note, comments need to be approved before they are published.