a_adli/colouring-montreal
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update load_csv python script

Browse Source 
- use building_id if present in CSV
- print DEBUG even if response from API is 200
- handle empty-string sust_dec (which fails because empty-string isn't a valid
  enum value as defined in 011.sustainability.up.sql) by deleting from data
  if present
This commit is contained in:
Tom Russell 2019-11-21 13:17:17 +00:00
parent 2ffa14cdbb
commit f3b9be39bf
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
etl/join_building_data/load_csv.py
View File

@ -47,9 +47,14 @@ def main(base_url, api_key, source_file):
if building_id is None: if building_id is None:
continue continue
if 'sust_dec' in line and line['sust_dec'] == '':
del line['sust_dec']
response_code, response_data = update_building(building_id, line, api_key, base_url) response_code, response_data = update_building(building_id, line, api_key, base_url)
if response_code != 200: if response_code != 200:
print('ERROR', building_id, response_code, response_data) print('ERROR', building_id, response_code, response_data)
else:
print('DEBUG', building_id, response_code, response_data)
def update_building(building_id, data, api_key, base_url): def update_building(building_id, data, api_key, base_url):
@ -64,6 +69,9 @@ def update_building(building_id, data, api_key, base_url):
def find_building(data, base_url): def find_building(data, base_url):
if 'building_id' in data:
return data['building_id']
if 'toid' in data: if 'toid' in data:
building_id = find_by_reference(base_url, 'toid', data['toid']) building_id = find_by_reference(base_url, 'toid', data['toid'])
if building_id is not None: if building_id is not None: