Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
July 19, 2021 05:23 pm GMT

Installing Scrapy in Fedora34 with pypy( pypy3 )

Pre-context:

The thing is I love fast things. And I am sure the reader also pursue same nature like me. Hence, pypy3 is the choice.
Fedora is nice but for me it is new and new thing is not always hassle free so to make your life better this instructions will be life saving.

First install pypy3 and respective pip pkgmanager:

sudo dnf install pypy3wget https://bootstrap.pypa.io/get-pip.py -o get-pip.pypypy3 get-pip.py

Then install the below dependencies which are the root cause of scrapy installation failures. So do it carefully.

sudo dnf install pypy3-develsudo dnf install libxml2-develsudo dnf install libxslt-develsudo dnf install @development-toolssudo dnf install libffi-develinstall them if you find them: [zlib1g-dev, libssl-dev] (optional)

Now create virtual environment:

pypy3 -m venv venv

Activate the virtual environment:

source venv/bin/activate

Now install scrapy:

pypy3 -m pip install setuptoolspypy3 -m pip install scrapy

Hopefully scrapy will be installed successfully.
Now create scrapy project.

scrapy startproject demo

Create a spider:

cd demoscrapy genspider demo demo

In fine, this is the way to have our beloved scrapy working in fedora.
If you have any problem you can comment below. I will try my best to resolve it.


Original Link: https://dev.to/amitkumarmalaker/installing-scrapy-in-fedora34-with-pypy-pypy3-f4m

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To