Skip to content

Commit 6ebc039

Browse files
Wukkkinz-0725Wukkkinz-0725
committed
feat: add NASA api
1 parent f67c95e commit 6ebc039

File tree

8 files changed

+396
-0
lines changed

8 files changed

+396
-0
lines changed

‎README.md

+109
Original file line numberDiff line numberDiff line change
@@ -2485,6 +2485,115 @@ df.head(1)
24852485
| 1899 | Triple descent and the two kinds of overfittin... | [NeurIPS] | 2020 |
24862486
</details>
24872487

2488+
#### [NASA](api-connectors/nasa) -- Collect NASA Data.
2489+
<details>
2490+
<summary>What are the title of Astronomy Picture of the Day from 2020-01-01 to 2020-01-10?</summary>
2491+
2492+
```python
2493+
from dataprep.connector import connect
2494+
2495+
# You can get ”nasa_access_key“ by following https://api.nasa.gov/
2496+
conn_nasa = connect("api-connectors/nasa", _auth={'access_token': nasa_access_key})
2497+
2498+
df = await conn_nasa.query("apod", start_date='2020-01-01', end_date='2020-01-10')
2499+
df['title']
2500+
```
2501+
| id | title |
2502+
|---:|:--------------------------------|
2503+
| 0 | Betelgeuse Imagined |
2504+
| 1 | The Fainting of Betelgeuse |
2505+
| 2 | Quadrantids over the Great Wall |
2506+
|... | ... |
2507+
| 9 | Nacreous Clouds over Sweden |
2508+
</details>
2509+
2510+
<details>
2511+
<summary>What are Coronal Mass Ejection(CME) data from 2020-01-01 to 2020-02-01?</summary>
2512+
2513+
```python
2514+
from dataprep.connector import connect
2515+
2516+
# You can get ”nasa_access_key“ by following https://api.nasa.gov/
2517+
conn_nasa = connect("api-connectors/nasa", _auth={'access_token': nasa_access_key})
2518+
2519+
df = await conn_nasa.query('cme', startDate='2020-01-01', endDate='2020-02-01')
2520+
df
2521+
```
2522+
2523+
| id | activity_id | catalog | start_time | ... | link |
2524+
|---:|:----------------------------|:------------|:------------------|:----|:---------------------------------------------------------|
2525+
| 0 | 2020-01-05T16:45:00-CME-001 | M2M_CATALOG | 2020-01-05T16:45Z | ... | https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/CME/15256/-1 |
2526+
| 1 | 2020-01-14T11:09:00-CME-001 | M2M_CATALOG | 2020-01-14T11:09Z | ... | https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/CME/15271/-1 |
2527+
| .. | ... | ... | ... | ... | ... |
2528+
| 4 | 2020-01-25T18:54:00-CME-001 | M2M_CATALOG | 2020-01-25T18:54Z | ... | https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/CME/15296/-1 |
2529+
</details>
2530+
2531+
<details>
2532+
<summary>How many Geomagnetic Storms(GST) have occurred from 2020-01-01 to 2021-01-01? When is it?</summary>
2533+
2534+
```python
2535+
from dataprep.connector import connect
2536+
2537+
# You can get ”nasa_access_key“ by following https://api.nasa.gov/
2538+
conn_nasa = connect("api-connectors/nasa", _auth={'access_token': nasa_access_key})
2539+
2540+
df = await conn_nasa.query('gst', startDate='2020-01-01', endDate='2021-01-01')
2541+
print("Geomagnetic Storms have occurred %s times from 2020-01-01 to 2021-01-01." % len(df))
2542+
df['start_time']
2543+
```
2544+
Geomagnetic Storms have occurred 1 times from 2020-01-01 to 2021-01-01.
2545+
2546+
| id | start_time |
2547+
|---:|:--------------------------------|
2548+
| 0 | 2020-09-27T21:00Z |
2549+
</details>
2550+
2551+
<details>
2552+
<summary>How many Solar Flare(FLR) have occurred and completed from 2020-01-01 to 2021-01-01? How long did they last?</summary>
2553+
2554+
```python
2555+
import pandas as pd
2556+
from dataprep.connector import connect
2557+
2558+
# You can get ”nasa_access_key“ by following https://api.nasa.gov/
2559+
conn_nasa = connect("api-connectors/nasa", _auth={'access_token': nasa_access_key})
2560+
2561+
df = await conn_nasa.query('flr', startDate='2020-01-01', endDate='2021-01-01')
2562+
df = df.dropna(subset=['end_time']).reset_index(drop=True)
2563+
df['duration'] = pd.to_datetime(df['end_time']) - pd.to_datetime(df['begin_time'])
2564+
print('Solar Flare have occurred %s times from 2020-01-01 to 2021-01-01.' % len(df))
2565+
print(df['duration'])
2566+
```
2567+
There are 1 times Geomagnetic Storms(GST) have occurred from 2020-01-01 to 2021-01-01.
2568+
2569+
| id | duration |
2570+
|---:|:------------------------------|
2571+
| 0 | 0 days 01:07:00 |
2572+
| 1 | 0 days 00:23:00 |
2573+
| 2 | 0 days 00:47:00 |
2574+
2575+
</details>
2576+
2577+
<details>
2578+
<summary>What are Solar Energetic Particle(SEP) data from 2019-01-01 to 2021-01-01?</summary>
2579+
2580+
```python
2581+
import pandas as pd
2582+
from dataprep.connector import connect
2583+
2584+
# You can get ”nasa_access_key“ by following https://api.nasa.gov/
2585+
conn_nasa = connect("api-connectors/nasa", _auth={'access_token': nasa_access_key})
2586+
2587+
df = await conn_nasa.query('sep', startDate='2019-01-01', endDate='2021-01-01')
2588+
df
2589+
```
2590+
2591+
| id | sep_id | event_time | instruments | ... | link |
2592+
|---:|:----------------------------|:------------------|:--------------------------------|:----|:-------------------------------------------------------- |
2593+
| 0 | 2020-11-30T04:26:00-SEP-001 | 2020-11-30T04:26Z | ['STEREO A: IMPACT 13-100 MeV'] | ... | https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/SEP/16166/-1 |
2594+
| 1 | 2020-11-30T14:16:00-SEP-001 | 2020-11-30T14:16Z | ['STEREO A: IMPACT 13-100 MeV'] | ... | https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/SEP/16169/-1 |
2595+
2596+
</details>
24882597

24892598
### Shopping
24902599

‎api-connectors/nasa/_meta.json

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"tables": [
3+
"apod",
4+
"flr",
5+
"cme",
6+
"sep",
7+
"gst"
8+
]
9+
}

‎api-connectors/nasa/apod.json

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
{
2+
"version": 1,
3+
"request": {
4+
"url": "https://api.nasa.gov/planetary/apod",
5+
"method": "GET",
6+
"authorization": {
7+
"type": "QueryParam",
8+
"keyParam": "api_key"
9+
},
10+
"params": {
11+
"date": false,
12+
"start_date": false,
13+
"end_date": false,
14+
"count": false,
15+
"thumbs": false
16+
}
17+
},
18+
"response": {
19+
"ctype": "application/json",
20+
"tablePath": "$[*]",
21+
"schema": {
22+
"copyright": {
23+
"target": "$.copyright",
24+
"type": "string"
25+
},
26+
"date": {
27+
"target": "$.date",
28+
"type": "string"
29+
},
30+
"explanation": {
31+
"target": "$.explanation",
32+
"type": "string"
33+
},
34+
"hdurl": {
35+
"target": "$.hdurl",
36+
"type": "string"
37+
},
38+
"media_type": {
39+
"target": "$.media_type",
40+
"type": "string"
41+
},
42+
"title": {
43+
"target": "$.title",
44+
"type": "string"
45+
},
46+
"url": {
47+
"target": "$.url",
48+
"type": "string"
49+
}
50+
},
51+
"orient": "records"
52+
}
53+
}

‎api-connectors/nasa/cme.json

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
{
2+
"version": 1,
3+
"request": {
4+
"url": "https://api.nasa.gov/DONKI/CME",
5+
"method": "GET",
6+
"authorization": {
7+
"type": "QueryParam",
8+
"keyParam": "api_key"
9+
},
10+
"params": {
11+
"startDate": true,
12+
"endDate": true
13+
}
14+
},
15+
"examples": {
16+
"start_date": "'2020-01-01'",
17+
"end_date": "'2020-02-01'"
18+
},
19+
"response": {
20+
"ctype": "application/json",
21+
"tablePath": "$[*]",
22+
"schema": {
23+
"activity_id": {
24+
"target": "$.activityID",
25+
"type": "string"
26+
},
27+
"catalog": {
28+
"target": "$.catalog",
29+
"type": "string"
30+
},
31+
"start_time": {
32+
"target": "$.startTime",
33+
"type": "string"
34+
},
35+
"source_location": {
36+
"target": "$.sourceLocation",
37+
"type": "string"
38+
},
39+
"active_region_num": {
40+
"target": "$.activeRegionNum",
41+
"type": "string"
42+
},
43+
"note": {
44+
"target": "$.note",
45+
"type": "string"
46+
},
47+
"instruments": {
48+
"target": "$.instruments[*].displayName",
49+
"type": "object"
50+
},
51+
"linked_events": {
52+
"target": "$.linkedEvents[*]",
53+
"type": "object"
54+
},
55+
"link": {
56+
"target": "$.link",
57+
"type": "string"
58+
}
59+
},
60+
"orient": "records"
61+
}
62+
}

‎api-connectors/nasa/flr.json

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
{
2+
"version": 1,
3+
"request": {
4+
"url": "https://api.nasa.gov/DONKI/FLR",
5+
"method": "GET",
6+
"authorization": {
7+
"type": "QueryParam",
8+
"keyParam": "api_key"
9+
},
10+
"params": {
11+
"startDate": true,
12+
"endDate": true
13+
}
14+
},
15+
"examples": {
16+
"start_date": "'2020-01-01'",
17+
"end_date": "'2021-01-01'"
18+
},
19+
"response": {
20+
"ctype": "application/json",
21+
"tablePath": "$[*]",
22+
"schema": {
23+
"flr_id": {
24+
"target": "$.flrID",
25+
"type": "string"
26+
},
27+
"instruments": {
28+
"target": "$.instruments[*].displayName",
29+
"type": "string"
30+
},
31+
"begin_time": {
32+
"target": "$.beginTime",
33+
"type": "string"
34+
},
35+
"peak_time": {
36+
"target": "$.peakTime",
37+
"type": "string"
38+
},
39+
"end_time": {
40+
"target": "$.endTime",
41+
"type": "string"
42+
},
43+
"class_type": {
44+
"target": "$.classType",
45+
"type": "string"
46+
},
47+
"source_location": {
48+
"target": "$.sourceLocation",
49+
"type": "string"
50+
},
51+
"active_region_num": {
52+
"target": "$.activeRegionNum",
53+
"type": "string"
54+
},
55+
"linked_events": {
56+
"target": "$.linkedEvents[*]",
57+
"type": "object"
58+
},
59+
"link": {
60+
"target": "$.link",
61+
"type": "string"
62+
}
63+
},
64+
"orient": "records"
65+
}
66+
}

‎api-connectors/nasa/gst.json

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
{
2+
"version": 1,
3+
"request": {
4+
"url": "https://api.nasa.gov/DONKI/GST",
5+
"method": "GET",
6+
"authorization": {
7+
"type": "QueryParam",
8+
"keyParam": "api_key"
9+
},
10+
"params": {
11+
"startDate": true,
12+
"endDate": true
13+
}
14+
},
15+
"examples": {
16+
"start_date": "'2020-01-01'",
17+
"end_date": "'2021-01-01'"
18+
},
19+
"response": {
20+
"ctype": "application/json",
21+
"tablePath": "$[*]",
22+
"schema": {
23+
"gst_id": {
24+
"target": "$.gstID",
25+
"type": "string"
26+
},
27+
"start_time": {
28+
"target": "$.startTime",
29+
"type": "string"
30+
},
31+
"allKpIndex": {
32+
"target": "$.allKpIndex[*]",
33+
"type": "object"
34+
},
35+
"linked_events": {
36+
"target": "$.linkedEvents[*]",
37+
"type": "object"
38+
},
39+
"link": {
40+
"target": "$.link",
41+
"type": "string"
42+
}
43+
},
44+
"orient": "records"
45+
}
46+
}

0 commit comments

Comments
 (0)