An extractor running for six months will eventually fail. That is the nature of the exercise: it depends on a site it does not control.
So the real question is not whether it will break, but how long it takes anyone to notice.
1. The site changes its layout
The most frequent cause, by a long way. A block moves, a class is renamed, and the extractor looks for an element that no longer exists.
The trap is that it does not necessarily stop. It returns an empty field and collection carries on as though nothing happened.
2. Pagination changes
The site moves from page navigation to infinite scroll, or caps how many results can be reached. The collection still runs. It now brings back a fraction of the catalogue, and it says nothing about it.
3. A new defence appears
A captcha, stricter behaviour checks, a blocked address range. This one is loud: collection stops dead. Paradoxically it is the most comfortable to deal with, because you see it immediately.
4. The content moves without the page moving
A currency changes. A date format flips from day-month to month-day. A decimal separator differs by region.
The data arrives, it looks clean, it is wrong. This is the most expensive failure, because it surfaces downstream, once decisions have been made on it.
5. Volume
A catalogue doubles, an execution window becomes too short, a quota is reached. The degradation is gradual and looks at first like nothing more than a slowdown.
What makes all of this manageable
Monitoring, and nothing else. A healthy extractor produces a volume, a fill rate and a distribution of values that vary little from one day to the next.
Watching those three indicators reveals almost every failure above before a user notices. Including the silent ones, which are the worst.
So we treat maintenance as part of the service rather than an option. A collection is not delivered the day it works, but the day you know it has stopped working. Our method sets out how we go about it.