Commit Graph

130 Commits

Author SHA1 Message Date
Tom Russell
79724cc449 Copy from stdin (cat user-accessible file) when loading geometries 2018-09-30 21:23:19 +01:00
Tom Russell
2a1902f6ce Update ETL docs 2018-09-29 18:29:57 +01:00
Tom Russell
73aa3df290 Clip OSMM to GLA on extraction 2018-09-27 21:37:47 +01:00
Tom Russell
322a976f7a Fix get_test_polygons script 2018-09-25 22:01:09 +01:00
Tom Russell
342167f9c9 Update UPRN-load script 2018-09-25 21:47:58 +01:00
Tom Russell
d9797385d9 Use default pool size (CPU count) in ETL 2018-09-25 21:47:29 +01:00
Tom Russell
3f9c9f3221 Split indexing further, UPRN requires bigint 2018-09-25 21:46:22 +01:00
Tom Russell
bddd7e769f Rename etl scripts 2018-09-25 20:46:16 +01:00
Tom Russell
c6b3d3d5ca Extract and load addressbase/mastermap using id, gml, parallel 2018-09-25 19:20:41 +01:00
Tom Russell
181e850225 Parallel extract/filter OS data 2018-09-21 11:10:39 +01:00
Tom Russell
f6f7cc1341 Save test polygons projected 2018-09-10 10:44:09 +01:00
Tom Russell
204740e46e Sketch matching data by best-intersection 2018-09-09 11:58:50 +01:00
Tom Russell
5695f2dc9d Record python packages for etl 2018-09-09 11:34:37 +01:00
Tom Russell
54633d9e04 Create building-per-geometry 2018-09-09 11:32:27 +01:00
Tom Russell
3711fc5f80 Load test polygons 2018-09-09 11:32:12 +01:00
Tom Russell
981d3608a2 Mappings for Camden and Fitrovia CASA data 2018-08-01 17:06:00 +01:00
Tom Russell
69bb6c5790 Provide transform when joining data 2018-08-01 17:05:02 +01:00
Tom Russell
847eb75a70 Fix loading buildings 2018-08-01 15:48:31 +01:00
Tom Russell
990e4241cf Sketch generic update from shapes 2018-08-01 14:17:10 +01:00
Tom Russell
1c30f4d436 Skip trying to join buildings/geometries up front 2018-08-01 13:50:08 +01:00
Tom Russell
c7b322f214 Skip duplicate geometries 2018-08-01 13:49:16 +01:00
Tom Russell
f6c8323bfa Load buildings from CSV 2018-08-01 13:49:03 +01:00
Tom Russell
cd65a5aaab Load from geojson to db 2018-08-01 13:12:56 +01:00
Tom Russell
39160d8a09 Join each polygon to UPRNs if possible 2018-07-27 17:47:22 +01:00
Tom Russell
92b81928ca Rename etl scripts to reflect stages 2018-07-27 15:48:00 +01:00
Tom Russell
105271a1e8 Sketch loading Addressbase buildings to database
- TODO handle intersection with geometries
2018-07-17 09:10:24 +01:00
Tom Russell
9316cfb719 Load OSMM geometries to database 2018-07-17 09:09:13 +01:00
Tom Russell
62b65f44f9 Extract OS MasterMap shell 2018-05-30 20:18:37 +01:00
Tom Russell
dd72ccd108 Extract OS AddressBase to CSV 2018-05-30 18:35:15 +01:00
Tom Russell
7c97aba950 Script comments for OS data extract 2018-05-30 14:31:23 +01:00