Skip to content Skip to sidebar Skip to footer

Scrapy Only Returns First Result

I'm trying to scrape preformatted html seen here. But my code only returns 1 price instead of all 10 prices. Code seen here: class MySpider(BaseSpider): name = 'working1' a

Solution 1:

You are getting the first result of the xpath match. Instead, iterate over all of them:

for price in sel.xpath("//text()[contains(., '$')]").extract():
    print price.strip(r"\r\n\t")

Prints (there are multiple occurrences of $0.03):

$0.03
$0.03
$0.03
$0.03
$0.03
$0.03
$0.03
$0.03
$0.03
$0.03

Post a Comment for "Scrapy Only Returns First Result"