Scraping infrastructure
Scheduled pipelines
Repeatable runs with failure alerts.
ops.reckap.com/scrape/pipelines
Collect, schedule + rate
Reckap scrape deskScheduleFetchStore
| Limit | Value |
|---|---|
| Req / min | Capped in scope |
| Retry | Backoff + alert |
Stay inside documented rate limits.
Cron-style runs with logs and a page when a job fails or stalls.
Repeatable runs with failure alerts.
Problems we hear, outcomes we ship
Buyer problems
- Jobs are started by hand with no stall alert.
- Failures are silent until the dataset is empty.
Outcomes
- Scheduled runs with logs.
- Fail/stall alerts with owners.
What is included
- Schedule
- Logs
- Fail alert
- Owner
How this work is shaped
Same desk language as the parent lane - scoped to this package.
ops.reckap.com/scrape/pipelines
Pipeline
CronRunAlert
Repeat runs + fail alerts.
How we deliver
Click a step - the desk below updates. That panel is a preview, not another step.
Step preview
01, Schedule
ops.reckap.com/scrape/allowlist
Allowlist, sources
Reckap scrape desk| Source | robots/ToS | Note |
|---|---|---|
| Public catalog | Allow | Rate capped |
| Login area | Block | Out of scope |
| Listings feed | Allow | Fields limited |
No credential stuffing. Public allowlist only.
More in this lane
Other packages under Scraping infrastructure.
FAQ
Next step
Ready to talk scope?
Bring the job, the systems you already run, and who owns the handoff. We write the rest.