ops.reckap.com/ethical-scraping/compliance/robots-tos
robots / ToS
robotsToSMemo
Written per source.
Ethical scraping and data extraction
Source review before a collector is built.
Review, go / no-go
Reckap scrape desk| Check | Result |
|---|---|
| robots.txt | Go |
| ToS scrape clause | Go |
| Auth wall | No-go |
Written per source before any collector ships.
Go/no-go reviews for robots.txt, ToS, and privacy practices before any collector ships - so ethics is not an afterthought.
Source review before a collector is built.
Pick a package - tables and flows show the shape of the work.
Click a step - the desk below updates. That panel is a preview, not another step.
Step preview
01, Review
Allowlist, sources
Reckap scrape desk| Source | robots/ToS | Note |
|---|---|---|
| Public catalog | Allow | Rate capped |
| Login area | Block | Out of scope |
| Listings feed | Allow | Fields limited |
No credential stuffing. Public allowlist only.
Next step
Ready to talk scope?
Bring the job, the systems you already run, and who owns the handoff. We write the rest.