Compliance and ethics advisory
robots.txt and ToS review
Written go/no-go per source.
ops.reckap.com/scrape/robots
Review, go / no-go
Reckap scrape desk| Check | Result |
|---|---|
| robots.txt | Go |
| ToS scrape clause | Go |
| Auth wall | No-go |
Written per source before any collector ships.
Per-source memo: robots.txt, ToS scrape clauses, and a clear go or no-go before build.
Written go/no-go per source.
Problems we hear, outcomes we ship
Buyer problems
- robots.txt is skimmed, not written into the scope.
- ToS bans are discovered after a complaint.
Outcomes
- Memo per source with go/no-go.
- Out-of-scope list for blocked sources.
What is included
- robots check
- ToS notes
- Go/no-go
- Out-of-scope list
How this work is shaped
Same desk language as the parent lane - scoped to this package.
ops.reckap.com/scrape/robots
robots / ToS
robotsToSMemo
Written per source.
How we deliver
Click a step - the desk below updates. That panel is a preview, not another step.
Step preview
01, robots
ops.reckap.com/scrape/allowlist
Allowlist, sources
Reckap scrape desk| Source | robots/ToS | Note |
|---|---|---|
| Public catalog | Allow | Rate capped |
| Login area | Block | Out of scope |
| Listings feed | Allow | Fields limited |
No credential stuffing. Public allowlist only.
More in this lane
Other packages under Compliance and ethics advisory.
FAQ
Next step
Ready to talk scope?
Bring the job, the systems you already run, and who owns the handoff. We write the rest.