Skip to content

Commit 109da60

Browse files
committed
Added new get_images() function
1 parent 7fcefc8 commit 109da60

File tree

4 files changed

+40
-2
lines changed

4 files changed

+40
-2
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,5 @@ venv
22
.idea
33
example.py
44
parents.csv
5-
children.csv
5+
children.csv
6+
images.csv

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,10 @@ children = scraper.get_variants(parents)
2727
children.to_csv('children.csv', index=False)
2828
print('Children: ', len(children))
2929

30+
31+
images = scraper.get_images(parents)
32+
images.to_csv('images.csv', index=False)
33+
print('Images: ', len(images))
34+
3035
```
3136

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
setup(
99
name='ShopifyScraper',
1010
packages=['shopify_scraper'],
11-
version='0.001',
11+
version='0.002',
1212
license='MIT',
1313
description='ShopifyScraper is a Python package that scrapes Shopify products and variants to Pandas dataframes.',
1414
long_description=long_description,

shopify_scraper/scraper.py

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,35 @@ def get_variants(products):
108108
df_variants = df_variants.merge(df_parent_data, left_on='product_id', right_on='parent_id')
109109
return df_variants
110110

111+
112+
def json_list_to_df(df, col):
113+
"""Return a Pandas dataframe based on a column that contains a list of JSON objects.
114+
115+
Args:
116+
df (Pandas dataframe): The dataframe to be flattened.
117+
col (str): The name of the column that contains the JSON objects.
118+
119+
Returns:
120+
Pandas dataframe: A new dataframe with the JSON objects expanded into columns.
121+
"""
122+
123+
rows = []
124+
for index, row in df[col].iteritems():
125+
for item in row:
126+
rows.append(item)
127+
df = pd.DataFrame(rows)
128+
return df
129+
130+
131+
def get_images(df_products):
132+
"""Get images from a list of products.
133+
134+
Args:
135+
df_products (pd.DataFrame): Pandas dataframe of products from get_products()
136+
137+
Returns:
138+
images (pd.DataFrame): Pandas dataframe of images
139+
"""
140+
141+
return json_list_to_df(df_products, 'images')
142+

0 commit comments

Comments
 (0)