Beautiful Soup - Results To Csv For All Items In Lists

The below snippet 'works' but is only outputting the first record to the CSV. I'm trying to get it to output the same output, but for each gun in the list of gun urls in the all_l

Solution 1:

Try running the middle section this way:

for a_link in all_links:
    gun_label = []
    gun_span = []

    res = requests.get(a_link)
    soup = bs(res.content, 'html.parser') #note it's 'res.content', not just 'res'
    for gun_details in soup.select('div.gunDetails'):
        for l in gun_details.select('label'):
            gun_label.append(l.text.replace(':',''))
    for s in gun_details.select('span'):
        gun_span.append(s.text)

    #this block is now indented differently - it's INSIDE the 'for' loop
    my_dict = dict(zip(gun_label, gun_span))
    with open('mycsvfile.csv', 'a') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=None)
        for key in my_dict.keys():
            csvfile.write(f"{key},{my_dict[key]}\n")