Medium Article CloudFlare Blocking #49
Description
Describe the bug or technical issue
Seems when I run the scrapper, the output in qdrant is:
Just a moment...medium.comVerify you are human by completing the action below.medium.com needs to review the security of your connection before proceeding.Verification successfulWaiting for medium.com to respond...Enable JavaScript and cookies to continue Ray ID: 90122c119b67c434Performance & security by Cloudflare
To Reproduce
Steps to reproduce the behavior:
Simply running the web scrapper and letting it run through the pipeline
Expected behavior
Ideally free articles that are open and viewable without a sub or an account should work
Screenshots
.
Desktop (please complete the following information):
- Browser chromium